Touch  Lab  Report  8 


Multimodal  Virtual  Environments: 

MAGIC  Toolkit  and  Visual-Haptic  Interaction  Paradigms 

I-Chun  Alexandra  Hou  and  Mandayam  A.  Srinivasan 
RLE  Technical  Report  No.  620 

January  1998 


Sponsored  by 

Naval  Air  Warfare  Center  Training  Systems  Division 


N6 1 339-96-K-0002 
Office  of  Naval  Research 
N000 14-97-1 -063 5 


The  Research  Laboratory  of  Electronics 
MASSACHUSETTS  INSTITUTE  OF  TECHNOLOGY 
CAMBRIDGE,  MASSACHUSETTS  02139-4307 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

JAN  1998  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-01-1998  to  00-01-1998 

4.  TITLE  AND  SUBTITLE 

Multimodal  Virtual  Environments:  MAGIC  Toolkit  and  Visual-Haptic 
Interaction  Paradigms 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Massachusetts  Institute  of  Technology, The  Research  Laboratory  of 
Electronics, 77  Massachusetts  Avenue, Cambridge, MA, 02139-4307 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

18.  NUMBER  19a.  NAME  OF 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

unclassified  unclassified  unclassified 

104 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Multimodal  Virtual  Environments:  MAGIC  Toolkit  and  Visual-Haptic 

Interaction  Paradigms 
by 

I-Chun  Alexandra  Hou 


Submitted  to  the  Department  of  Mechanical  Engineering 
on  August  26,  1996,  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 
Master  of  Science  in  Mechanical  Engineering 


Abstract 

The  MAGIC  Toolkit  is  an  application  program  and  library  file  that  allows  users  to  see, 
manually  feel,  create,  edit,  and  manipulate  objects  in  the  virtual  environment.  Using  the 
PHANToM  haptic  interface,  a  user  can  build  a  complex  virtual  object  or  scene  by  adding 
object  primitives  to  the  virtual  workspace.  Object  primitives  are  pre-programmed  objects, 
such  as  a  cylinder  and  a  sphere,  that  have  visual  and  haptic  characteristics  which  can  be 
modified  with  a  touch  to  the  virtual  menu  wall.  Using  the  MAGIC  Toolkit  is  a  simple  way 
to  create  multimodal  virtual  environments  without  directly  writing  the  program  code  or 
creating  the  environment  in  another  application  and  then  translating  the  file.  The  library 
file  has  many  useful  routines  for  manipulating  the  virtual  scene  for  the  creation  of  a  spe¬ 
cific  end  application.  The  MAGIC  Toolkit  with  extensions  is  useful  for  many  applications 
including  creation  of  environments  for  training,  prototyping  structures  or  products,  devel¬ 
oping  standardized  motor  coordination  tests  to  monitor  patient  recovery,  or  entertainment. 
This  DOS-based  application  runs  on  a  single  Pentium  90  MHz  processor  that  computes 
the  haptic  updates  at  1500  Hz  and  the  graphic  updates  at  30  Hz.  Since  the  field  of  vir¬ 
tual  environments  is  still  fairly  new,  there  are  some  fundamental  questions  about  how  best 
to  interact  with  the  environment.  In  this  thesis,  experiments  on  visual-haptic  size  ratios, 
visual  scaling,  and  cursor  control  paradigms  have  been  conducted  to  investigate  user  pref¬ 
erence  and  performance.  These  experiments  also  investigate  the  role  of  vision  and  haptics 
in  navigating  through  a  maze.  Visual-haptic  size  ratios  refer  to  the  relative  size  of  the 
visual  display  to  the  haptic  workspace.  Visual  scaling  refers  to  the  effects  of  increasing  and 
decreasing  the  size  of  the  visual  display  relative  to  the  haptic  workspace.  Cursor  control 
paradigms  fall  into  two  categories:  position  control  and  force  control.  Results  of  the  exper¬ 
iments  find  that  subjects  prefer  large  visual-haptic  ratios,  small  haptic  workspaces,  and  a 
position  controlled  cursor.  Subjects  perform  best  with  a  large  visual  display  and  a  small 
haptic  workspace.  In  negotiating  a  maze,  subjects  perform  best  when  given  both  visual 
and  haptic  cues,  with  a  slight  decrease  in  performance  when  given  only  haptic  cues,  and 
with  a  significant  decrease  in  performance  when  given  only  visual  cues.  When  subjects  are 
trained  on  a  large  visual  size,  their  performances  improve  linearly  with  the  increase  in  visual 
display.  Subjects  perform  best  when  there  is  a  high  correlation  of  position  and  movement 
between  the  visual  and  haptic  workspaces  for  cursor  control. 

Thesis  Supervisor:  Mandayam  A.  Srinivasan 
Title:  Principal  Research  Scientist 
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Chapter  1 


Introduction 


1.1  Virtual  Environments 

Virtual  Environments  (VEs)  are  computer  generated  worlds  that  give  humans  a  means 
to  design  and  experience  events  that  would  otherwise  be  impossible,  difficult,  expensive, 
or  dangerous  in  a  real  environment.  Proposals  for  the  use  of  VEs  fall  into  four  main 
categories:  1)  teaching  and  training,  2)  health  care,  3)  design  and  manufacturing  ,  and  4) 
entertainment.  For  the  first  category,  VEs  allow  simulation  of  training  programs  such  as 
piloting  or  performing  surgery.  This  type  of  application  allows  potential  pilots  or  doctors 
to  practice  and  perfect  techniques  in  their  respective  fields  with  impunity  should  anything 
go  wrong.  The  potential  pilot  would  not  endanger  himself/herself,  any  passengers,  or  the 
aircraft  in  a  VE  simulation,  neither  w'ould  the  medical  student  endanger  the  life  of  a  patient. 
Training  in  an  artificial  environment  of  an  actual  or  hypothetical  situation  allows  the  person 
to  learn  the  correct  procedures  and  techniques  of  a  given  task. 

In  health  care,  VEs  could  potentially  diagnose  or  track  the  recovery  status  of  a  patient 
with  a  standardized  test  that  would  stimulate  and  record  specific  reactions.  In  the  commer¬ 
cial  industries  of  design  and  manufacturing,  VEs  could  be  used  to  design  and  test  structures 
or  products.  This  type  of  simulation  saves  on  time  and  materials  involved  in  constructing 
or  manufacturing.  In  the  entertainment  industry,  VEs  can  simulate  imaginative  scenarios 
for  people  to  play  in.  The  quality  of  a  VE  can  be  measured  based  on  how  “immersed”  a 
person  feels.  If  a  VE  can  deceive  the  human  senses  into  believing  that  the  environment 
he/she  is  in  is  real,  the  person  will  feel  immersed  in  the  environment. 

Humans  have  five  primary  senses  to  perceive  their  surroundings:  sight,  sound,  touch, 
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smell,  and  taste.  The  three  main  modalities  humans  use  to  interact  with  and  navigate 
through  the  real  world  are  sight,  sound,  and  touch.  The  human  vision  and  audition  sys¬ 
tems  are  purely  sensory  in  nature;  in  contrast,  the  human  haptic  system,  which  includes 
the  human  sense  of  touch,  can  both  sense  and  act  on  the  environment  [Srinivasan,  1994]. 
There  has  been  a  great  deal  of  research  about  the  human  visual  and  auditory  system.  Facts 
discovered  about  these  modes  of  perception  have  aided  the  development  of  visual  and  audio 
interfaces.  The  availability  of  visual  and  audio  interfaces  coupled  with  computer  control 
and  technology  allow  for  the  rapid  progress  of  these  aspects  in  the  design  of  VEs.  Com¬ 
puter  graphics  has  evolved  to  a  state  where  images  presented  has  an  uncanny  likeness  to 
a  real  object.  Audio  devices  can  now  output  sounds  with  amazing  fidelity  to  the  original 
environment  in  which  the  sound  is  recorded.  Compared  to  what  is  known  of  the  human 
vision  and  audition,  understanding  of  human  haptics  is  still  very  limited,  yet  the  ability  to 
haptically  explore  and  manipulate  objects  is  what  greatly  enhances  the  sense  of  immersion 
in  VEs[Srinivasan,  1994]. 

Haptics,  in  the  context  of  VEs,  have  two  intrinsically  linked  categories:  human  hap¬ 
tics  and  machine  haptics.  The  development  of  machine  haptics  allow  for  experiments  on 
human  haptic  abilities  and  limits.  By  knowing  human  haptic  abilities  and  limits,  haptic 
interfaces  can  be  improved  and  designed  to  enhance  the  sense  of  touch.  Figure  1-1  depicts 
the  categories  of  haptics  and  the  relationship  between  human  haptics  and  machine  haptics. 


Figure  1-1:  Haptics  Tree 
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1.2  Human  Haptics 


The  study  of  human  haptics  has  two  aspects:  physiological  and  perceptual.  The  goal  of 
physiological  haptics  is  to  understand  the  biomechanical  and  neural  aspects  of  how  tactual 
sensory  signals  as  well  as  motor  commands  are  generated,  transmitted,  and  processed.  The 
goal  of  perceptual  haptics  is  to  understand  how  humans  perceive  -with  the  tactual  sense:  the 
methods  and  levels  of  accuracy  for  detection,  discrimination,  and  identification  of  various 
stimuli. 

Human  tactual  sensing  can  be  divided  into  two  sensory  modes,  kinesthetic  and  tac¬ 
tile.  Kinesthetic  refers  to  the  sensing  of  position,  movement,  and  orientation  of  limbs  and 
the  associated  forces  with  the  sensory  input  originating  from  the  skin,  joints,  muscles, 
and  tendons.  Tactile  sensing  refers  to  the  sense  of  contact  with  an  object.  This  type  of 
sensing  is  mediated  by  the  responses  of  low-threshold  mechanoreceptors  near  the  area  of 
contact  [Srinivasan,  1994],  The  tactual  sensing  in  combination  with  the  human  motor  appa¬ 
ratus  in  the  haptic  system  allow  humans  to  use  their  hands  to  perceive,  act  on,  and  interact 
with  their  environment. 

Quantitative  research  has  discovered  several  facts  about  the  human  haptic  system: 


•  Humans  can  distinguish  vibration  frequencies  up  to  1  KHz  through  the  tactile  sense. 

•  Humans  can  detect  joint  rotations  of  a  fraction  of  a  degree  performed  over  about  a 
second. 

•  The  bandwidth  of  the  kinesthetic  system  is  estimated  to  be  20-30  Hz. 

•  The  JND  (Just  Noticeable  Difference)  for  the  finger  joint  is  about  2.5  degrees,  for  the 
wrist  and  elbow  is  2  degrees,  and  about  0.8  degrees  for  the  shoulder. 

•  A  stiffness  of  at  least  25  N/mm  is  needed  for  an  object  to  be  perceived  as  rigid  by 
human  observers.  [Tan  et.  al.,  1994] 

•  The  JND  is  20%  for  mass,  12  %  for  viscosity,  7%  for  force,  and  8%  for  compli- 
ance[Beauregard,  1996]. 

In  addition  to  finding  out  how  humans  react  to  different  stimuli,  how  they  perform  with 
different  interfaces,  and  how  they  react  in  different  environments,  insight  into  what  feels 
natural  to  them  and  what  types  of  interfaces  may  be  suitable  for  different  tasks  is  also 
needed.  Understanding  human  haptic  abilities  and  limitations  can  lead  to  improvements  of 
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current  haptic  devices  and  the  development  of  new  devices  which  will  give  the  user  a  more 
immersive  experience. 

1.3  Machine  Haptics 

The  development  of  machine  haptics  is  composed  of  hardware  development  and  software 
development.  Haptic  interfaces  allow  humans  to  interact  with  the  computer.  This  interac¬ 
tion  requires  a  physical  device  to  transmit  the  appropriate  stimuli  and  software  to  control 
the  stability  and  desired  action  and  reaction. 

1.3.1  Haptic  Hardware  Development 

There  are  three  categories  of  haptic  interfaces:  tactile  displays,  body  based  devices,  and 
ground  based  devices[reviewed  by  Srinivasan,  1994].  Tactile  displays  stimulate  the  skin 
surface  to  convey  tactile  information  about  an  object.  Research  into  this  area  has  primarily 
focused  on  conveying  visual  and  auditory  information  to  deaf  and  blind  individualsfBach- 
y-Rita,  1982],  Body  based  devices  are  exoskeletal  in  nature.  They  could  be  flexible,  such  as 
a  glove  or  a  suit  worn  by  the  user,  or  they  could  be  rigid,  such  as  jointed  linkages  affixed  to 
the  user.  One  such  device  is  the  “Rutgers  Master  II”,  which  uses  four  pneumatic  cylinders 
with  linear  position  sensors  in  addition  to  a  rotary  sensor  to  determine  the  location  of  the 
fingers  and  actuate  a  desired  force[Gomez,  Burdea,  Langrana,  1995]. 

Ground  based  devices  include  joysticks  and  hand  controllers.  One  of  the  first  force- 
reflecting  hand  controllers  was  developed  at  the  University  of  North  Carolina  with  the 
project  GROPE,  a  7  DOF  manipulator  [Brooks  et  al.,  1990].  Margaret  Minsky  developed 
the  Sandpaper  System,  a  2-DOF  joystick  with  feedback  forces  that  simulates  textures. 
[Minsky  et  al.,  1990  ]  The  University  of  British  Columbia  developed  a  6  DOF  magnetically 
levitated  joystick  which  features  low-inertia  and  low  friction  [Salcudean,  1992].  MIT’s 
Artificial  Intelligence  Laboratory  developed  the  PHANToM.  It  features  three  active  degrees 
of  freedom  and  three  passive  degrees  of  freedom  with  a  point  contact  which  has  low  inertia 
and  high  bandwidth[Massie  and  Salisbury,  1994]. 

This  thesis  discusses  the  development  of  a  software  application  designed  to  be  used  with 
the  PHANToM,  but  can  be  applied  to  any  point-interaction  haptic  interface  device  which 
outputs  a  force  given  a  position. 
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1.3.2  Haptic  Software  Development 

The  development  of  haptic  interfaces  has  resulted  in  a  need  for  increased  understanding 
of  the  human  haptic  system.  The  growth  of  this  field  has  also  found  some  problems  and 
limitations  in  the  performance  of  haptic  devices.  Due  to  the  inherent  nature  of  haptics,  all 
computations  must  be  calculated  in  real-time.  Given  the  fact  the  VEs  are  enhanced  with  the 
combination  of  visual,  auditory,  and  haptic  stimuli,  a  substantial  amount  of  computational 
power  is  required  to  run  a  multi-modal  VE  in  real-time.  The  development  of  efficient  code 
and  methods  of  rendering  in  the  three  main  interactive  modalities  is  essential  for  a  quality 
simulation.  Since  motors  can  only  generate  a  finite  amount  of  torque  over  certain  periods  of 
time,  methods  of  rendering  scenes  which  will  give  the  illusion  of  a  stiff  surface  are  needed. 
The  software  development  can  possibly  compensate  for  hardware  limitations  and  make  the 
virtual  world  feel  more  natural.  Since  the  virtual  world  does  not  have  to  obey  all  the  laws 
of  the  physical  world,  software  development  can  also  create  effects  that  are  not  possible  in 
a  real  environment. 

Studies  on  the  software  requirements  for  stiff  virtual  walls  have  been  conducted  at 
Northwestern  University  [Colgate,  1994].  It  is  possible  for  a  user  to  touch  one  side  of  a 
thin  object  and  be  propelled  out  the  opposite  side,  because  surfaces  are  usually  rendered 
using  an  algorithm  which  output  a  force  proportional  to  the  amount  of  penetration  into 
a  surface.  This  motivated  the  development  of  a  constraint  based  algorithm  which  keeps 
a  history  of  the  cursor’s  surface  contact  and  outputs  the  force  in  a  direction  normal  to 
the  contact  surface[Zilles  and  Salisbury,  1995].  Displaying  a  deformable  object  gives  the 
user  an  illusion  of  a  soft  object[Swarup  1995].  This  method  of  rendering  compensates  for 
a  device’s  motor  torque  limit,  since  the  visual  presentation  of  a  deformed  object  implies 
an  intentional  non-stiff  object.  A  study  in  visual  dominance  has  found  that  when  a  user  is 
presented  with  two  virtual  springs  and  asked  to  determine  which  of  the  two  is  stiffer,  the 
user  will  almost  always  choose  the  spring  that  visually  compresses  less  for  a  given  force  and 
ignore  the  haptic  dependent  cues[Srinivasan  et.  al.,  1996].  Force  shading  is  a  method  that 
maps  a  pre-specified  radial  vector  to  a  planar  surface  in  order  to  create  the  haptic  illusion 
of  a  curved  surface  when  a  planar  surface  is  displayed[Morgenbesser  and  Srinivasan,  1996]. 
This  method  is  useful  in  creating  complex  curved  objects.  One  could  render  a  polyhedral 
mesh  that  describes  an  angular  object,  add  force  shading,  and  create  a  perception  of  a 
smooth  curved  object.  This  would  reduce  computation  time  since  it  is  simpler  to  specify  a 
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polyhedral  approximation  to  a  curved  surface  than  it  is  to  specify  a  continuously  smooth 
complex  object. 

With  the  development  of  haptic  interfaces  comes  the  development  of  software  for  use 
with  the  device.  First  basic  algorithms  need  to  be  developed  to  control  the  device.  Next 
it  must  be  able  to  render  virtual  objects  or  scenes  accurately.  Once  these  basic  needs  are 
satisfied,  the  device  can  be  used  in  a  higher  level  application.  To  facilitate  the  end  goal, 
it  would  be  useful  to  have  a  development  environment  to  create  virtual  scenes.  This  thesis 
describes  the  development  of  a  software  toolkit  to  facilitate  the  creation  of  multimodal 
virtual  environments. 

Increased  understanding  of  human  haptics,  improved  rendering  techniques,  and  better 
haptic  interfaces  in  combination  with  visual  and  auditory  developments  will  allow  multi¬ 
modal  virtual  environments  to  reach  a  state  where  complex  applications  such  as  surgical 
training  can  be  realized. 


1.4  Contributions  to  Multimodal  Virtual  Environments 

The  goals  of  this  thesis  are  to  develop  applications  and  investigate  interaction  variations 
that  would  help  in  the  expansion  of  the  use  of  multimodal  virtual  environments.  Key 
factors  that  play  a  role  in  how  quickly  and  easily  a  new  field,  such  as  multimodal  virtual 
environments,  becomes  widespread,  are  cost  and  ease  of  use. 

A  system  which  is  capable  of  rendering  high  quality  multimodal  virtual  environments  will 
most  likely  be  very  expensive.  The  intrinsic  nature  of  this  immersive  technology  requires 
real-time  updates  of  the  visual,  haptic,  and  audio  environment.  The  updates  require  a 
significant  amount  of  computing  power.  For  the  graphics  rendering,  an  usual  setup  is  to 
have  a  Silicon  Graphics  machine  or  a  PC  with  a  graphics  accelerator  running  3-dimensional 
scenes  generated  with  Openlnventor.  This  type  of  system  commonly  costs  at  least  $10,000. 

The  physical  hardware  of  a  haptic  device  is  also  needed  for  manual  interaction  with  the 
virtual  environments.  A  device,  such  as  the  PHANToM,  costs  about  $20,000.  In  addition, 
computational  power  is  required  to  interpret  the  location  and  control  the  force  feedback. 
Depending  on  the  computational  power  of  the  graphics  systems  described  above,  the  haptic 
computations  can  be  on  the  same  system,  or  may  require  a  separate  processor.  The  necessity 
of  another  processor  adds  to  the  cost  of  the  complete  system.  The  same  arguments  can  be 
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applied  to  the  addition  of  the  audio  aspect  of  the  multimodal  VE. 

A  high  quality  multimodal  VE  rendering  system  can  very  quickly  become  very  expensive. 
There  are  several  applications  of  VEs  which  do  not  require  the  highest  fidelity  of  performance 
in  all  sensory  modes.  In  this  thesis,  the  goal  is  to  develop  an  application  which  focuses  on 
high  fidelity  haptics  and  adequate  graphics  for  a  single  processor  system.  This  basic  type  of 
VE  rendering  system  allows  for  the  fundamental  studies  on  the  human  haptic  system  and  on 
human  interaction  with  multimodal  VEs.  This  system  is  relatively  simple  and  inexpensive; 
it  requires  only  a  PC  and  a  haptic  interface  device,  such  as  the  PHANToM. 

To  make  such  a  system  easy  to  use,  the  MAGIC  Toolkit  has  been  developed.  It  includes 
an  application  program  and  a  set  of  library  files  that  allows  an  user  to  easily  create  3-D 
haptic  and  2-D  visual  environments.  The  application  has  object  primitives  in  which  the 
user  can  use  like  building  blocks  to  create  a  scene.  These  objects  have  attributes  such  as 
size,  location,  stiffness,  and  color,  which  can  be  readily  be  changed  with  a  touch  to  the 
menu. 

This  “Building  Blocks”  type  of  application  makes  the  creation  of  multimodal  VEs  simple 
even  for  the  novice  user,  and  affordable.  The  availability  of  an  effective  and  affordable 
system,  increases  the  viability  of  the  growing  use  of  multimodal  VEs.  A  large  base  of  users 
creates  a  platform  in  which  more  applications  can  be  created  and  a  better  understanding 
of  interactions  can  be  achieved. 

In  addition  to  the  development  of  the  MAGIC  Toolkit,  this  thesis  also  describes  the 
use  of  the  Toolkit  in  creating  mazes  for  a  series  of  human  visual-haptic  interaction  exper¬ 
iments.  These  experiments  study  the  performance  and  preference  of  users  when  different 
size  visual  and  haptic  displays  are  presented.  Other  parameters  that  are  varied  include 
different  objectives  for  completing  the  maze,  different  levels  sensory  feedback,  and  different 
cursor  control  paradigms.  In  some  experiments  the  subjects  are  told  to  optimize  speed, 
in  others,  to  optimize  accuracy.  Some  experiments  varied  the  size  of  both  the  visual  and 
the  haptic  display,  while  other  experiments  varied  only  the  size  of  the  visual  display.  The 
sensory  feedback  experiments  consist  of  three  sessions  in  which  the  subjects  are  presented  at 
first  with  both  visual  and  haptic  feedback,  then  with  only  haptic  feedback,  and  finally  with 
only  visual  feedback.  Another  set  of  experiments  investigates  the  effects  of  cursor  control 
differences  between  position  control  and  force  control. 

The  larger  goal  of  this  project  is  to  make  multimodal  VEs  simple,  effective,  easy  to  use, 
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and  affordable  so  that  it  can  be  incorporated  into  many  applications.  This  project  also  aims 
to  achieve  a  better  understanding  of  the  human  visual-haptic  interaction. 

The  following  list  summarizes  the  contributions  made  in  this  thesis  to  the  field  of  mul¬ 
timodal  virtual  environments: 

•  developed  the  MAGIC  Toolkit,  a  VE  building  blocks  application  and  library  file  for 
a  single  Pentium  processor  PC  system  which  has  both  visual  and  haptic  rendering. 

•  developed  a  menu  driven  program  to  be  a)  user  friendly,  and  b)  easy  to  change  at¬ 
tributes  of  objects. 

•  developed  an  organized  structure  for  object  characteristics  that  users  can  easily  access 
and  add  information  about  the  attributes  of  the  object. 

•  developed  a  novel  rendering  algorithm  that  allows  for  a  speedy  calculation  of  forces 
for  a  cone. 

•  defined  various  human  visual-haptic  interactions. 

•  conducted  experiments  to  study  the  effects  of  visual  and  haptic  size  on  user  preference 
and  performance. 

•  conducted  experiments  to  study  the  effects  of  visual  and  haptic  feedback  on  user 
preference  and  performance. 

•  defined  various  cursor  control  paradigms. 

•  conducted  experiments  to  study  the  effects  of  various  cursor  control  paradigms  on 
user  preference  and  performance. 

•  found  that  subjects  perform  best  with  and  prefer  a  large  visual  workspace  paired  with 
a  smaller  haptic  workspace. 

•  found  that  subjects  perform  best  with  both  visual  and  haptic  feedback. 

•  found  that  subjects  prefer  position  control  cursor  paradigms  to  force  control  cursor 
paradigms. 

•  found  that  an  illusion  of  having  stiffer  walls  can  be  created  using  a  haptic  workspace 
that  is  larger  than  the  visual  workspace. 
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1.5  Overview 


To  help  the  reader  with  the  organization  of  this  thesis,  the  following  is  a  summary  of  what 
is  presented  in  each  chapter. 

•  Chapter  2  discusses  the  development  of  the  MAGIC  Toolkit.  This  DOS-based  toolkit 
facilitates  the  creation  and  editing  of  complex  virtual  scenes  and  complex  virtual 
objects.  A  description  of  how  object  primitives  are  used  to  build  the  scene  is  followed 
by  a  discussion  of  the  various  characteristics  of  each  object.  An  innovative  way  to 
render  a  cone  is  described.  A  description  of  the  library  files  is  given. 

•  Chapter  3  discusses  several  visual-haptic  interaction  paradigms.  Size  ratios  of  the 
visual  workspace  to  the  haptic  workspace  and  their  effects  on  users’  perception  of 
the  environment  are  investigated.  One  set  of  variations  has  a  combination  of  two 
visual  workspace  sizes  and  two  haptic  workspace  sizes.  The  other  set  of  variations  has 
one  haptic  workspace  size  and  four  visual  workspace  sizes.  Cursor  control  is  another 
important  aspect  of  user  interaction.  Four  different  types  of  cursor  control  paradigms 
are  discussed  including  two  position  control  and  two  force  control  cursor  paradigms. 

•  Chapter  4  describes  the  experimental  methods  used  to  investigate  the  effects  of  dif¬ 
ferent  visual-haptic  interaction  paradigms.  One  set  of  experiments  investigates  user 
preference  and  performance,  given  different  visual-haptic  workspace  ratios.  It  also  in¬ 
vestigates  the  performance  when  given  different  objectives  for  completing  the  maze,  for 
example,  speed  vs.  accuracy.  The  role  of  sensory  feedback  is  also  investigated.  Sub¬ 
jects  were  presented  with  the  maze  with  both  haptic  and  visual  feedback,  with  haptic 
feedback  but  without  visual  feedback,  and  without  haptic  feedback  but  with  visual 
feedback.  Another  set  of  experiments  investigated  training  effects.  The  performance 
of  subjects  who  trained  on  a  large  visual  workspace  is  compared  with  the  performance 
of  subjects  who  trained  on  a  small  visual  workspace.  The  former  describes  the  effects 
of  decreasing  visual  scaling,  the  latter  describes  the  effects  of  increasing  visual  scaling. 
The  final  experiment  tested  subjects  on  the  performance  and  preference  of  the  cursor 
control  paradigms.  In  this  set,  they  were  given  a  single  visual  size  that  corresponded 
to  the  haptic  size. 
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Chapter  5  presents  the  results  of  the  experiments.  Subjects  prefer  and  perform  best  on 
a  large  visual  display  and  a  small  haptic  display.  Results  show  that  subjects  perform 
best  when  given  both  visual  and  haptic  feedback.  Their  performance  decreased  by 
26%  when  given  only  haptic  feedback,  but  decreased  over  61%  when  given  only  visual 
feedback.  In  the  visual  scaling  experiments,  subjects  performed  consistently  when  they 
trained  on  a  large  visual  display.  They  performed  less  consistently  when  they  trained 
on  a  small  visual  display.  In  the  cursor  paradigm  experiment,  subjects  preferred 
position  control  over  force  control  paradigms.  They  also  completed  the  maze  faster 
with  the  position  control  paradigms. 

Chapter  6  discusses  the  significance  of  the  results  of  the  experiments.  Subjects  prefer 
and  perform  best  on  a  large  visual  environment  with  a  small  haptic  environment. 
Presenting  a  small  visual  environment  coupled  with  a  large  haptic  environment  gives 
the  illusion  of  very  stiff  walls.  Having  both  visual  and  haptic  feedback  gives  rise  to 
the  best  performance.  When  only  one  sensory  mode  is  given,  performance  is  bet¬ 
ter  with  only  haptic  feedback  than  with  only  visual  feedback.  Training  on  a  visual 
environment  larger  than  the  haptic  environment  results  in  a  linear  improvement  in 
time  performance  when  the  visual  environment  is  increased  as  the  haptic  environment 
remains  at  the  same  size.  There  is  a  limit  to  the  improvement  in  time  performance 
when  a  subject  is  trained  on  small  visual  and  haptic  environment.  In  fact,  the  perfor¬ 
mance  of  some  subjects  actually  degrade  at  larger  visual-haptic  size  ratios.  Subjects 
find  position  control  cursor  paradigms  easier  than  force  control.  Performance  is  better 
when  there  is  a  high  correlation  in  motion  and  force  between  the  visual  and  haptic 
realms. 

Chapter  7  concludes  with  an  evaluation  of  the  application  toolkit  and  experiments. 
It  also  discusses  directions  for  future  work.  The  sample  size  of  subjects  for  the  ex¬ 
periments  is  small,  but  representative.  This  study  shows  the  general  trends  of  per¬ 
formance.  Continuing  this  study  with  more  subjects  could  more  accurately  specify 
the  degree  to  which  these  trends  are  true.  It  would  also  be  interesting  to  conduct  a 
similar  series  of  experiments  with  much  smaller  haptic  workspaces  to  study  human 
fine  motor  control. 


Chapter  2 


The  MAGIC  Toolkit 


2.1  Motivation 


The  current  methods  of  creating  virtual  environments  axe  not  very  user  friendly,  especially 
to  users  who  are  not  familiar  with  the  field  of  haptics.  These  methods  require  the  user  to 
manually  program  the  specific  shapes,  sizes,  and  locations  of  the  objects,  or  to  draw  the 
desired  virtual  scene  in  another  application,  such  as  CAD  or  FEA,  then  have  a  program  that 
translates  the  file  into  a  form  that  is  suitable  for  a  haptic  display.  These  time  consuming 
and  user  un- friendly  methods  prompted  the  development  of  the  MAGIC  Toolkit,  a  software 
application  program  and  library  which  would  allow  users  to  easily  create  and  edit  complex 
virtual  objects  or  scenes.  This  virtual  “building  blocks”  program  is  easy  to  use  for  both 
the  low  level  user  and  the  high  level  user.  The  novice  can  use  the  menu  driven  program  as 
a  creation  medium.  This  user  can  add  objects  to  the  scene  to  view  and  touch.  The  high 
level  user  has  a  goal  of  using  the  scenes  created  in  the  menu  driven  program  for  a  complex 
application.  This  user  can  employ  the  library  of  functions  to  help  in  the  manipulation  of 
the  scene  in  the  programming. 

The  MAGIC  Toolkit  has  a  collection  of  object  primitives  that  the  user  can  employ  to 
create  VEs  or  complex  virtual  objects.  It  organizes  the  visual  and  haptic  characteristics 
of  objects  in  a  structure  which  facilitates  the  visual  and  haptic  presentation  of  the  VE. 
Designed  to  be  used  with  the  PHANToM  haptic  interface  device,  the  MAGIC  Toolkit 
allows  the  user  to  see,  manually  feel,  create,  and  edit  a  virtual  environment. 
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2.2  Apparatus 


The  MAGIC  Toolkit  is  designed  to  be  used  with  a  point  interaction,  open  loop  control 
haptic  interface  that  outputs  a  force  for  a  given  position.  The  PHANToM,  shown  in  Figure 
2.1,  has  three  active  degrees  of  freedom  (x,  y,  z)  and  three  passive  degrees  of  freedom  (9, 
(j),  ip).  The  stylus  at  the  end  of  linkage  is  a  pen-like  device  that  the  user  holds  to  explore 
the  haptic  workspace.  The  MAGIC  Toolkit  is  a  DOS-based  application  written  in  Borland 
C++.  Its  routines,  however,  are  transportable  to  other  domains  with  a  minimal  amount  of 
revision.  It  was  a  conscious  decision  to  write  the  application  based  in  DOS.  This  application 
would  not  have  to  share  processor  time  with  other  applications,  such  as  the  ones  running  in 
Windows.  This  results  in  a  higher  bandwidth  of  operations  since  the  processor  is  devoted  to 
only  one  application.  The  trade  off  for  using  DOS  is  the  limited  number  of  colors  available 
and  the  lack  of  3-dimensional  graphics  rendering  routines.  Therefore,  the  MAGIC  Toolkit 
comprises  of  a  2-dimensional  visual  display  and  a  3-dimensional  haptic  workspace.  The 
haptic  control  loop  update  frequency  for  this  program  is  approximately  1500  Hz.  This  is 
the  performance  when  running  on  a  90  MHz  Pentium  processor.  It  will  of  course  have  a 
higher  bandwidth  with  a  faster  processor. 


Figure  2-1:  PHANToM  Haptic  Interface 
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2.3  Modes  of  Operation 


The  MAGIC  Toolkit  has  several  modes  of  operation.  First,  it  allows  the  user  to  feel  and 
explore  the  environment.  Second,  it  allows  the  user  to  move  the  selected  object  in  the  scene 
by  touching  the  selected  object  and  pushing  it  around.  Third,  it  allows  the  user  to  add 
objects  to  the  scene  and  edit  the  features  of  the  objects. 

2.4  Coordinate  System 

The  coordinate  system  of  this  application  is  centered  in  the  middle  of  the  haptic  workspace. 
The  x  axis  is  on  the  horizontal  plane  starting  at  the  center  and  pointing  to  the  right.  The 
y  axis  is  on  the  horizontal  plane  starting  at  the  center  and  pointing  forward  and  away  from 
the  user.  The  z  axis  is  on  the  vertical  plane  starting  at  the  center  and  pointing  up.  Figure 
2-2  shows  a  diagram  of  the  coordinate  system. 


Figure  2-2:  The  Coordinate  System 


2.5  Object  Primitives 

Object  primitives  are  pre-programmed  objects  that  have  visual  and  haptic  characteristics 
that  can  be  modified  to  create  a  virtual  scene.  The  object  primitives  in  the  MAGIC  Toolkit 
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include  a  sphere,  cylinder,  cone,  cube,  and  rectangular  prism.  When  the  user  touches  an 
object  with  the  PHANToM,  the  user  will  feel  a  contact  force  appropriate  for  the  object. 
The  contact  forces  are  calculated  the  simple  linear  spring  law, 

F  =  —kx  (2.1) 

The  force,  F,  is  proportional  to  the  amount  of  indentation,  x.  The  indentation,  x,  is 
the  amount  of  penetration  into  the  object  from  the  surface.  The  force  is  directed  in  the 
opposite  direction  of  the  indentation  vector.  The  following  is  a  description  of  how  each  of 
these  primitives  is  constructed. 

2.5.1  Sphere 

The  sphere  is  haptically  and  visually  defined  by  a  3-dimensional  centerpoint  and  a  radius.  It 
is  one  of  the  simpler  objects  to  render.  All  the  force  vectors  point  radially  outward  from  the 
centerpoint.  Figure  2-3a  shows  the  3-dimensional  sphere.  Figure  2-3b  shows  a  cross-section 
of  the  sphere  with  the  force  vector  directions. 


a 

Figure  2-3: 
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Sphere 


2.5.2  Cylinder 

The  cylinder  is  defined  by  a  3-dimensional  centerpoint,  a  length,  a  radius,  and  an  axis  of 
orientation.  It  is  composed  of  three  surfaces.  The  top  and  bottom  surfaces  are  defined  as 
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planes  with  constraints  at  the  circumference  of  the  circle.  When  the  user  touches  these 
surfaces,  the  contact  force  returned  is  normal  to  the  surface.  The  third  surface  is  the  body 
of  the  cylinder.  All  the  force  vectors  for  the  body  of  the  cylinder  point  radially  outward 
from  the  central  axis.  The  central  axis  is  the  line  through  the  centerpoint  pointing  in  the 
same  direction  as  the  axis  of  orientation.  Near  the  intersection  of  the  body  and  the  planar 
surface,  the  forces  are  defined  by  the  location  of  the  cursor.  Of  the  two  force  vectors  that 
may  apply,  the  one  of  lessor  magnitude  is  returned.  Figure  2-4a  shows  the  3-dimensional 
cylinder  with  the  key  attributes.  Figure  2-4b  shows  a  cross-section  of  the  cylinder  with  the 
force  vector  directions  associated  with  each  region. 


a 
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Figure  2-4:  Cylinder 


2.5.3  Cone 

The  cone  is  defined  by  a  3-dimensional  centerpoint,  height,  and  base  radius.  The  centerpoint 
is  located  at  the  center  of  the  base  of  the  cone,  as  shown  in  Figure  2-5a. 

The  cone  is  composed  of  two  surfaces,  the  body  and  the  base.  The  force  vectors  for 
the  body  point  radially  outward  from  the  central  axis.  The  central  axis  is  the  line  passing 
through  the  centerpoint  and  the  vertex  of  the  cone  in  the  z-axis  direction.  Currently,  the 
cone  has  only  one  orientation.  The  base  of  the  cone  is  a  planar  surface  constrained  by 
the  circumference  of  the  circle  defined  by  the  base  radius.  The  force  vectors  for  the  base 
are  directed  along  normals  to  the  base  surface.  The  rendering  method  of  the  cone  does 
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Figure  2-5:  Cone 

not  depict  a  true  cone  since  the  force  vectors  returned  for  the  body  of  the  cone  are  not 
perpendicular  to  the  surface.  They  are  rather,  perpendicular  to  the  central  axis.  This  is  a 
simple  rendering  algorithm,  requiring  very  few  calculations,  but  still  creates  a  cone  that  is 
haptically  indistinguishable  from  one  that  has  a  force  vector  normal  to  all  surfaces.  One 
limitation  of  this  rendering  algorithm  is  the  difficulty  in  feeling  the  vertex  of  the  cone. 
Figure  2-5b  shows  the  horizontal  cross-section  of  the  cone  with  the  associated  force  vector 
directions. 

Near  the  intersection  of  the  conical  and  planar  surfaces,  the  force  vector  with  the  lesser 
magnitude  is  returned.  Figure  2-5c  shows  the  vertical  cross-section  of  the  cone  with  the 
respective  force  vectors  for  each  of  the  surfaces. 

2.5.4  Cube 

The  cube  is  defined  by  a  3-dimensional  centerpoint  and  the  length  of  one  side  as  shown 
in  Figure  2-6a.  It  is  composed  of  six  perpendicular  planar  surfaces.  The  force  fed  back  is 
based  on  the  location  of  the  cursor.  A  square  cross-section  is  essentially  divided  into  four 
triangular  regions  by  drawing  the  diagonals  as  shown  in  Figure  2-6b.  Each  of  the  triangular 
regions  has  an  associated  force  in  a  corresponding  planar  direction.  If  the  cursor  is  within 
the  region,  the  force  vector  is  in  the  direction  normal  to  the  surface  of  the  cube.  Now,  in  the 
three  dimensions,  the  cube  is  divided  into  six  tetrahedral  volumes.  In  each  of  the  volumes, 
the  force  vector  will  always  point  in  the  direction  normal  to  the  outer  surface. 
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Figure  2-6:  Cube  and  Cross-sectional  View  with  Force  Vectors 


2.5.5  Rectangular  Prism 

The  rectangular  prism  is  defined  by  a  3-dimensional  centerpoint,  length,  width,  and  height 
as  shown  in  Figure  2-7a.  The  prism  is  similar  to  the  cube,  differing  only  in  the  values  for 
the  length,  width,  and  height.  Figure  2- 7b  shows  the  cross-sectional  view  of  the  rectangular 
prism  with  the  associated  force  vectors  for  each  surface. 
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Figure  2-7:  Rectangular  Prism 
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2.6  Functions 


2.6.1  Location  Variation 

The  X,  Y,  and  Z  centerpoint  location  of  each  object  can  be  changed  in  increments  of  0.1 
inch  using  the  menu  bar. 

2.6.2  Size  Variation 

The  parameters  available  for  changing  the  size  of  an  object  include  length,  width,  height, 
and  radius.  The  user  can  change  the  values  of  each  of  these  parameters  in  increments  of  0.1 
inches.  When  the  parameter  is  not  applicable  for  the  selected  object,  for  example  a  radius 
for  the  cube,  the  value  is  not  incremented. 

2.6.3  Stiffness  Variation 

The  stiffness  of  the  object  has  an  initial  value  of  0.1.  It  can  be  changed  in  increments  of 
0.01  and  has  a  range  of  0  to  0.2. 

2.6.4  Color  Variation 

The  colors  available  to  chose  from  include:  Black,  Blue,  Green,  Cyan,  Red,  Magenta,  Brown, 
Light  Gray,  Dark  Gray,  Light  Blue,  Light  Green,  Light  Cyan,  Light  Red,  Light  Magenta, 
Yellow,  and  White.  These  are  the  16  colors  available  for  the  DOS  routines. 


2.7  User  Interface 

Figure  2-9  shows  the  visual  display  when  the  MAGIC  Toolkit  program  is  running.  There  is  a 
blue  background,  a  cursor,  two  buttons  indicating  the  feel  and  move  mode  of  operation,  two 
switches  that  allow  for  editing  of  the  workspace,  a  button  that  will  trigger  the  current  scene 
to  be  saved  into  a  file,  and  three  information  display  boxes  indicating  the  force  output,  the 
current  cursor  location,  and  the  centerpoint  of  the  selected  object.  All  buttons  and  switches 
are  haptically  located  on  the  vertical  front  wall  of  the  haptic  workspace.  A  touch  to  the 
region  of  the  switch  or  button  using  the  haptic  interface  device  will  trigger  the  respective 
action. 
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Figure  2-8:  Visual  Display  of  MAGIC  Working  Environment 


2.T.1  Modes  of  Operation 

There  are  two  black  buttons  located  symmetrically  at  the  top,  center  region  of  the  visual 
and  haptic  workspace.  One  is  the  FEEL  button,  located  on  the  right.  The  other  is  the 
MOVE  button,  located  on  the  left.  The  application  is  always  in  one  mode  or  the  other.  The 
active  mode  is  written  in  white,  while  the  inactive  mode  is  written  in  red.  The  FEEL  and 
MOVE  mode,  as  described  earlier,  allows  the  user  to  explore  and  manipulate  the  virtual 
environment,  respectively. 

2.7.2  Switches 

There  are  two  switches  visually  located  at  the  center  top  of  the  screen  one  above  the  other. 
Each  has  two  triangular,  red  incrementation  arrows  located  on  either  side  of  the  black  label 
area.  In  the  haptic  space,  the  switches  are  located  at  the  top  center  of  the  front  wall. 
When  the  user  touches  one  of  the  switches,  a  short  auditory  blip  signals  the  activation  of 
the  switch.  One  switch  toggles  the  parameters,  the  other  switch  toggles  the  values  of  the 
parameters.  The  parameters  include:  CENTER  X,  CENTER  Y,  CENTER  Z,  LENGTH, 
WIDTH,  HEIGHT,  RADIUS,  ADD,  SELECT,  COLOR,  and  STIFFNESS.  CENTER  X  is 
the  x  component  of  the  centerpoint  of  the  selected  object.  CENTER  Y  is  the  y  component 


of  the  centerpoint  of  the  selected  object.  CENTER  Z  is  the  z  component  of  the  centerpoint 
of  the  selected  object.  LENGTH,  WIDTH,  HEIGHT,  and  RADIUS  are  the  size  parameters 
of  the  selected  object.  If  the  parameter  is  not  applicable,  it  has  a  value  of  zero.  ADD  is  an 
editing  feature  that  allows  the  user  to  add  an  object  to  the  scene.  Objects  include  a  sphere, 
a  cone,  a  cylinder,  a  cube,  and  a  rectangular  prism. 

All  objects  have  an  initial  centerpoint  at  (x,  y,  z)  =  (0,  0,  0)  and  a  stiffness  value  of  0.1. 
The  dimensions  of  the  objects  and  locations  are  given  in  mils,  (0.001  inches).  The  initial 
sphere  has  a  radius  of  500  mils  and  an  original  color  of  red.  The  initial  cone  has  a  radius 
of  500  mils,  a  height  of  500  mils,  and  a  color  of  green.  There  are  three  initial  cylinders, 
one  along  each  axis,  that  all  have  a  radius  of  500  mils,  a  height  of  500  mils,  and  a  color 
of  green.  The  initial  cube  has  a  length  of  500  mils,  and  a  color  of  cyan.  There  are  three 
initial  rectangular  prisms  with  an  initial  color  of  red,  in  which  the  dimension  along  one  axis 
is  longer  than  the  dimensions  in  the  other  two.  One  has  an  initial  length  of  1000  mils,  a 
width  of  1000  mils,  and  a  height  of  500  mils.  The  other  has  an  initial  length  of  5000  mils, 
a  width  of  1000  mils,  and  a  height  of  500  mils.  The  last  has  an  initial  length  of  1000  mils, 
a  width  of  5000  mils,  and  a  height  of  500  mils. 

When  an  object  is  selected,  a  white  border  outlines  the  shape. 


2.7.3  Load/Save  Options 

When  the  program  first  starts,  it  prompts  the  user  for  the  file  he/she  wishes  to  load.  If  there 
is  such  a  file,  it  is  loaded;  if  not,  the  user  is  given  a  fresh  scene.  Once  the  user  has  edited  the 
file  or  created  a  new  scene,  there  is  a  save  button  located  at  the  top  right  corner  of  the  front 
wall.  The  user  is  then  prompted  to  enter  the  name  of  the  file  to  save  the  information.  Also, 
when  the  user  enters  “q”  from  the  keyboard,  the  same  prompt  will  appear  before  exiting 
the  program. 

The  file  that  the  information  is  saved  to  has  the  following  format.  It  first  saves  the 
number  of  objects  in  the  scene.  It  then  saves  the  information  about  each  object.  The 
information  saved  is  in  the  following  order:  object  number,  color,  radius,  length,  width, 
height,  x  centerpoint  location,  y  centerpoint  location,  z  centerpoint  location,  x  axis  of 
orientation  value,  y  axis  of  orientation  value,  z  axis  of  orientation  value,  stiffness,  and 
object  type. 
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2.7.4  Information  Display 


On  the  left  side  of  the  screen,  the  x,  y,  and  z  components  of  the  force  applied,  the  present 
cursor  position,  and  the  selected  object’s  center  location  are  displayed. 


2.8  Library  Files 

Associated  with  the  MAGIC  Toolkit  program,  is  a  set  of  library  files  that  defines  how  each 
object  and  its  features  are  associated.  Once  the  virtual  scene  is  saved  as  a  *.mgc  file,  it 
can  then  be  used  in  other  applications.  The  library  files  allow  the  scene  file  to  be  loaded, 
drawn,  and  manipulated  in  various  ways. 

One  of  the  library  files  defines  the  class  in  which  the  objects  are  organized  and  structured. 
Another  file  has  many  routines  useful  in  manipulating  the  scene.  The  following  describes 
the  functions. 


•  load-scene:  This  function  loads  the  scene  by  transferring  the  information  in  the  file 
and  assigning  it  to  the  objects.  It  also  returns  the  number  of  objects  in  the  scene. 

•  draw-scene:  This  function  draws  the  objects  in  the  scene  for  the  specified  point  of 
view,  either  from  the  x,  y,  or  z  axis. 

•  erase-scene:  The  function  visually  draws  the  objects  within  the  scene  in  the  color 
blue,  the  background  color.  This  in  essence  erases  the  scene. 

•  save-scene:  This  function  saves  the  object  characteristics  in  a  specified  file. 

•  scale-scene:  This  function  scales  the  objects  in  two  of  three  dimensions. 

•  change-color:  This  function  changes  the  color  of  the  specified  objects  to  the  desired 
color 

2.9  Evaluation 

Currently,  the  visuals  are  drawn  as  a  2-dimensional  projection.  It  is  at  times  difficult  to 
locate  an  object  in  a  3-dimensional  space  when  only  given  a  2-dimensional  view.  A  method 
used  to  help  locate  the  object  has  the  user  place  the  stylus  along  the  same  projectional 
space  as  the  object.  Visually  the  cursor  would  be  touching  or  inside  the  object.  The 
user  then  moves  the  stylus  along  the  axis  of  projection  until  the  object  is  found.  In  the 
future,  the  visuals  could  be  presented  as  a  3-dimensional  by  using  field  sequential  displays, 
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such  as  Crystal  Eyes,  or  a  shaded  representation  of  the  scene  to  give  the  appearance  of  a 
three  dimensional  view.  This  set  of  complex  graphics  will  require  more  processing  power 
due  to  the  inherent  nature  of  3-dimensional  graphics  to  be  run  in  real  time.  It  is  likely 
that  a  separate  processor  or  computer  devoted  to  visual  rendering  will  be  needed,  though 
advances  in  computer  technology  may  develop  a  processor  that  can  handle  both  the  3- 
dimensional  graphics  and  3-dimensional  haptics  rendering.  Another  avenue  is  to  develop 
efficient  graphics  rendering  algorithms  that  can  use  DOS  to  create  realistic  3-dimensional 
images.  Running  on  a  single  system  reduces  the  complexity  of  the  overall  structure.  It  also 
reduces  the  cost  of  the  operating  system. 

A  framework  developed  to  characterize  an  object’s  visual  and  haptic  features  is  easily 
expandable  to  include  other  features  such  as  texture,  mass,  density,  viscosity,  and  other 
material  properties.  An  ideal  Magic  Toolkit  would  include  dynamic  interactions  between 
the  objects  and  the  user.  In  other  words,  it  would  be  able  to  create  a  pin  joint,  a  ball  socket, 
and  a  sliding  joint.  This  would  allow  for  the  simulation  of  virtual  mechanisms. 

An  application  like  the  MAGIC  Toolkit  is  very  useful  in  creating  virtual  environments.  It 
provides  a  virtual  workspace  to  design,  create,  edit,  and  explore.  There  are  no  material  costs 
in  designing  in  this  medium.  Complex  geometries  can  be  modeled  easily.  A  complex  object 
can  be  created  using  simple  object  primitives  and  used  in  later  applications.  This  versatile 
Toolkit  can  create  training  environments  for  various  tasks,  create  standardized  environments 
to  test  and  record  reactions  and  evaluate  a  patient’s  progress,  model  structures  and  products 
to  test  for  viability,  robustness,  and  feasibility,  and  create  imaginative  environments  to  just 
have  fun  in,  or  develop  motor  coordination  skills. 

In  the  rest  of  this  thesis,  the  use  of  the  MAGIC  Toolkit  to  create  virtual  mazes  to  test 
human  performance  in  various  visual  haptic  paradigms  is  described.  These  experiments 
are  done  in  an  effort  to  understand  how  different  visual  and  haptic  cues  will  alter  human 
performance  and  to  find  what  type  of  visual-haptic  interaction  humans  prefer. 
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Chapter  3 


Visual-Haptic  Interactions 

3.1  Motivation 

Virtual  Environments  is  a  relatively  new  field.  Very  little  is  known  of  how  best  to  interact 
in  this  environment.  Because  this  field  is  still  in  its  infancy,  there  are  some  basic  questions 
to  be  answered  and  terminology  that  needs  to  be  defined.  Some  of  the  questions  are: 

1.  What  are  the  different  types  of  haptic- visual  interactions  that  are  possible  ?  In  terms 
of  relative  display  sizes?  In  terms  of  motion?  In  terms  of  force?  In  terms  of  background 
and  cursor  relationships? 

2.  What  type  of  relationship  SHOULD  the  haptic  and  visual  workspace  have? 

3.  In  what  situations  would  it  be  beneficial  to  have  one  type  of  interaction  over  another? 

4.  What  are  the  advantages  and  disadvantages  of  each  of  the  different  paradigms? 

5.  How  can  a  particular  implementation  of  the  haptic-visual  workspace  improve  the  sense 
of  immersion? 

6.  Which  type  of  haptic-visual  workspace  is  most  “comfortable”  and  “natural”  for  the 
user? 

7.  What  types  of  skills  and  motions  are  required  by  each  of  the  different  paradigms? 

8.  How  large  should  the  visuals  or  haptics  be?  How  large  does  the  visual  or  haptic 
workspace  need  to  be? 
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9.  How  feasible  is  it  to  emulate  large-scale  haptic  workspace  in  a  much  smaller  haptic 
workspace? 

10.  Can  a  small  haptic  workspace  be  mapped  to  a  perceptually  large  workspace  using 
visual  or  force  cues? 

This  chapter  on  Visual-Haptic  Interactions  will  define  some  of  the  modes  of  interaction. 
To  find  how  humans  act  and  react  to  these  paradigms  and  variations,  perceptual  experiments 
need  to  be  performed.  The  end  goal  is  to  find  what  type  of  interaction  paradigm  would  be 
comfortable  and  easy  to  use,  and  which  is  ideal  for  certain  tasks. 


3.2  Visual  and  Haptic  Size  Variations 


Since  the  physical  workspace  of  the  PHANToM  or  of  any  haptic  interface  is  limited,  it 
would  be  interesting  to  see  if  increasing  the  size  of  the  visual  display  would  cause  the  user 
to  perceive  a  larger  workspace.  It  would  also  be  interesting  to  find  the  effects  of  scaling 
one  modality  versus  the  other.  Enlarging  the  visual  display  is  analogous  to  looking  at  the 
workspace  through  a  magnifying  lens. 

A  set  of  experiments  was  designed  using  two  visual  workspaces  of  different  sizes  and 
two  similar  haptic  workspaces  of  different  sizes.  Pairing  each  visual  workspace  with  haptic 
workspace  results  in  the  matrix  shown  in  Table  3.1. 

Table  3.1:  Visual- Haptic  Size  Variations 


Haptic 

Size 

Visual  Size 

Large  Small 

Large 

Small 

Variation  1  Variation  2 
Variation  3  Variation  4 

Given  this  set  of  variations  it  would  be  interesting  to  see  how  subjects  perform  if  they 
are  told  to  minimize  time  versus  if  they  are  told  to  minimize  error,  where  an  error  is  defined 
as  any  contact  with  the  walls  of  the  maze.  It  would  also  be  interesting  to  see  the  variations 
in  their  performance  given  different  combinations  of  sensory  feedback.  A  subject  could  be 
presented  with  only  visual  feedback,  with  only  haptic  feedback,  or  with  both  haptic  and 
visual  feedback.  One  would  suspect  the  subjects  would  perform  best  if  encouraged  to  use 
as  many  senses  as  available.  However,  whether  they  would  perform  better  with  just  visual 
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feedback  or  with  just  haptic  feedback  is  unclear.  One  hypothesis  is  that  performance  would 
be  better  with  just  haptic  feedback  because  the  haptics  is  an  interactive  modality,  whereas 
vision  is  purely  sensory. 

Also,  the  performance  in  one  variation  versus  another  for  the  different  objectives  is 
also  unclear.  One  would  suspect  that  larger  haptics  would  have  fewer  errors  since  there 
is  physically  more  room  to  move  in,  and  that  larger  haptics  would  require  more  time  to 
complete  because  the  path  would  be  longer.  One  might  also  suspect  that  large  visuals  would 
have  a  better  performance  than  small  visuals.  Conducting  this  set  of  experiments  would 
give  conclusive  results  as  to  the  user  performance  and  preference  in  these  variations  and 
tasks. 


3.3  Visual  Scaling  Effects  on  Training 


One  might  also  wonder  about  the  effects  of  training  on  a  large  visual  size  versus  training 
on  a  small  visual  size.  If  there  are  a  number  of  visual  displays  differing  only  in  size,  and  a 
subject  is  presented  with  a  small  visual  workspace  for  training,  followed  by  presentations 
with  the  other  visual  sizes,  this  is  considered  an  increase  in  the  visual  scaling.  Similarly,  if 
the  subject  is  first  presented  with  the  largest  visual  workspace,  this  is  considered  a  decrease 
in  visual  scale.  In  general,  visual  scaling  is  defined  as  increasing  or  decreasing  the  visual 
display  size  throughout  a  given  task. 

One  would  suspect  that  larger  visuals  would  result  in  a  better  performance.  However, 
would  performance  always  improve  with  an  increase  in  visual  size?  Is  there  a  limit  in  which 
performance  would  level  off  or  even  decay?  Conducting  an  experiment  with  this  theme 
would  require  several  visual  size  displays  and  a  single  haptic  size  display.  One  might  also 
wonder  about  the  performance  in  these  variations  if  trained  on  a  large  visual  display  versus 
a  small  visual  display.  Training  on  a  small  visual  display  would  give  the  impression  of 
scaling  up  whereas  training  on  a  large  visual  display  would  give  the  impression  of  scaling 
down.  Results  of  this  experiment  would  tell  if  there  is  a  limit  to  visual  dominance  effect  and 
if  there  is  a  benefit  in  training  on  a  certain  size  display.  Table  3-2  lists  a  set  of  variations 
in  which  the  haptic  size  remains  constant  and  the  visual  size  increases. 
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Table  3.2:  Visual  Scaling  Variations 


Visual  Size 

Haptic  Size 

Variation  5 

Small 

Small 

Variation  6 

Medium 

Small 

Variation  7 

Large 

Small 

Variation  8 

Ex-large 

Small 

3.4  Cursor  Paradigms 

There  are  many  variations  in  which  a  user  can  interact  with  the  VE  using  a  haptic  device 
and  see  the  results  of  that  interaction.  The  cursor  and  the  scene  are  two  parameters  that 
can  be  varied  in  both  the  visual  and  haptic  domains.  The  cursor  can  be  stationary  or 
it  can  be  mobile.  The  scene  can  be  mobile  or  stationary.  Two  broad  categories  of  this 
interaction  is  in  terms  of  position  control  and  force  control.  For  position  control,  the 
haptic  cursor  is  moving.  For  force  control,  the  haptic  cursor  is  stationary.  The  display 
of  these  interactions  can  be  coupled  with  a  moving  scene  or  a  stationary  scene  in  both  the 
visual  and  haptic  environments.  A  more  detailed  description  of  the  cursor  paradigms  will 
explained  in  the  next  section.  Conducting  an  experiment  with  the  cursor  paradigms  will 
reveal  human  performance  in  these  different  environments.  This  will  give  insight  to  human 
reaction  in  these  various  virtual  environments,  so  that  in  the  future,  should  a  certain  task 
require  a  similar  human  control  of  the  virtual  environment,  there  will  be  an  understanding 
of  performance  and  preference.  Table  3-3  shows  the  different  characteristics  of  each  of  the 
cursor  control  paradigms. 


Table  3.3:  Cursor  Paradigms 


Analogy 

“mouse” 

“magnifying 

lens” 

“tele¬ 

video” 

“RC  car  ” 

Visual  Cursor 

moving 

static 

static 

moving 

Visual  Environment 

static 

moving 

moving 

static 

Haptic  Cursor 

moving 

static 

static 

Haptic  Environment 

static 

static 

moving 

Control  Type 

Position 

Control 

Position 

Control 

Force 

Control 

Force 

Control 
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3.4.1  Position  Control 


In  position  control,  the  motion  initiated  through  a  haptic  device  results  in  a  similar  move¬ 
ment  in  the  visual  workspace.  One  familiar  type  of  position  control  is  that  of  using  the 
computer  mouse.  When  the  mouse  moves  by  a  certain  amount  in  physical  space,  a  cursor 
moves  by  a  proportional  amount  in  the  visual  space.  This  “mouse”  type  of  position  control 
is  characterized  by  a  stationary  visual  environment  and  a  stationary  haptic  environment 
while  the  visual  and  haptic  cursors  are  moving  synchronously. 

An  alternative  method  of  position  control  is  analogous  to  exploring  an  environment  with 
a  “magnifying  lens” .  Imagine  a  person  holding  a  magnifying  lens  in  one  hand  and  move  it 
freely  to  view  locations  of  interest.  The  magnifying  lens  itself  is  the  haptic  cursor  moving 
in  a  static  haptic  environment.  As  the  person  looks  through  the  lens,  the  visual  moves  in 
a  direction  opposite  to  the  motion  of  the  lens,  while  the  visual  cursor  at  the  center  of  the 
lens  is  stationary  relative  to  the  lens  frame.  The  person  sees  a  visual  environment  moving 
to  the  left  when  the  magnifying  lens  is  moved  to  the  right.  The  “magnifying  lens”  type  of 
position  control  is  characterized  by  a  stationary  visual  cursor,  a  moving  haptic  cursor,  a 
stationary  haptic  environment,  and  a  moving  visual  environment. 

3.4.2  Force  Control 

In  force  control,  the  motion  initiated  by  the  user  results  in  a  force  felt  in  the  haptic  environ¬ 
ment,  and  the  force  applied  by  the  user  governs  the  cursor  location  and  motion  of  a  visual 
display.  The  force  applied  results  in  a  motion  proportional  to  the  velocity  of  the  element  in 
the  visual  space.  For  example,  a  video  game  joystick  moves  a  character  in  the  direction  of 
applied  force. 

There  are  two  types  of  force  control.  The  first,  which  is  called  “tele-video”,  has  a 
stationary  visual  and  haptic  cursor,  and  a  moving  visual  and  haptic  environment.  This  is 
analogous  to  a  video  camera  that  hangs  over  the  field  of  some  sports  stadiums.  This  is 
used  to  capture  a  close-up  view  of  the  athletes  during  the  game.  The  field  of  view  of  the 
camera  is  fairly  narrow,  but  the  camera  can  be  moved  to  a  desired  location.  An  editor  in 
a  production  room  controls  this  video  camera.  The  editor  can  see  the  athletes  on  the  field 
via  a  monitor  which  displays  the  field  of  view  of  the  video  camera.  When  the  editor  tries 
to  visually  follow  an  athlete  running  across  the  field,  he/she  applies  a  force  to  a  joystick 
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in  the  desired  direction.  The  athlete  appears  at  the  center  of  the  screen  and  the  visual 
environment  moves  in  a  direction  opposite  to  that  of  the  applied  force.  In  this  analogy,  the 
center  of  the  monitor  is  the  stationary  visual  cursor  and  when  the  editor  applies  a  force  to 
the  controller,  he/she  operates  stationary  haptic  cursor. 

The  second,  which  is  analogous  to  a  radio  controlled  toy  car,  has  a  stationary  haptic 
cursor,  a  moving  visual  cursor,  a  stationary  visual  environment,  and  a  moving  haptic  envi¬ 
ronment.  Imagine  a  person  maneuvering  a  radio  controlled  car  around  an  obstacle  course. 
The  person  is  standing  at  a  distance  watching  the  toy  car  move  as  he/she  applies  force  to 
the  controller  in  the  desired  direction.  The  car  is  the  moving  visual  cursor.  As  the  person 
watches,  he/she  sees  a  stationary  visual  environment.  The  joystick  in  the  radio  control  box 
is  the  stationary  haptic  cursor.  The  force  applied  to  the  controller  results  in  a  motion  of 
the  car.  This  paradigm  is  called  “RC  car”  for  short. 
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Chapter  4 


Experiments 


The  task  of  completing  a  maze  was  chosen  for  its  simplicity  and  complexity.  The  maze  was 
designed  to  have  a  single  path.  Multiple  paths  were  not  needed  since  the  subjects  were  not 
being  tested  on  their  cognitive  skills.  The  path  of  the  maze  had  various  sizes  to  change  the 
complexity  level.  It  also  had  many  turns  and  obstacles  in  the  path. 


4.1  Experimental  Procedure 

All  the  subjects  in  these  experiments  were  right  handed  and  between  20  to  28  years  of  age. 
At  the  start  of  each  session,  each  subject  was  asked  to  read  and  sign  a  consent  form  for 
participating  in  the  experiment  (see  Appendix  A).  A  short  demonstration  of  how  to  use  the 
PHANToM  preceded  the  start  of  the  experiment.  The  instructions  for  the  experiment  were 
presented  as  a  prompt  on  the  monitor.  Once  the  instructions  had  been  read,  the  subject’s 
understanding  of  the  procedure  was  verified. 

At  the  start  of  each  trial,  each  subject  was  asked  to  center  the  stylus  of  the  PHANToM. 
Two  cursors  representing  the  subject’s  3  dimensional  position  was  displayed.  One  cursor’s 
position  was  a  view  of  the  x-z  plane.  The  other  cursor’s  position  was  a  view  of  the  y-z  plane. 
When  the  subject  centered  both  cursors  in  the  middle  of  a  circle  on  the  screen,  the  stylus 
would  be  centered  in  the  haptic  workspace,  thus  ensuring  the  same  starting  position  for  each 
trial.  This  also  guaranteed  that  the  subject  did  not  start  the  maze  when  the  cursor  was 
within  an  object  which  would  result  in  an  unexpected  force  when  the  maze  was  displayed 
haptically. 

Once  the  subject  completed  the  maze,  he/she  was  asked  to  rate  the  difficulty  of  com- 
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pleting  the  maze  on  a  scale  of  1  to  5. 


•  1  =  very  hard 

•  2  =  hard 

•  3  =  medium  difficulty 

•  4  =  easy 

•  5  =  very  easy 


At  the  end  of  the  session,  each  subject  was  asked  to  fill  out  a  survey  about  the  strategy 
used  in  completing  the  maze,  the  differences  perceived  among  the  mazes  presented,  and 
their  preference  for  one  maze  over  another. 


4.2  Experimental  Design 

4.2.1  Apparatus 

These  experiments  were  conducted  with  T-Model  version  1.0  high  resolution  PHANToM 
haptic  interface  and  a  90  MHz  Pentium  Processor  running  in  DOS.  The  subject  sat  in 
a  chair  approximately  30  inches  away  from  a  NEC  MultiSync  5FGe  17  inch,  0.28  pitch 
monitor.  Since  all  the  subjects  were  right-handed,  the  PHANToM  was  located  to  the  right 
of  the  subject  with  an  arm  rest  to  support  the  arm  during  the  experiment.  The  keyboard 
was  located  in  front  of  the  subject  and  used  to  start  and  rate  the  trials  of  the  maze.  The 
experimental  apparatus  is  shown  in  Figure  4-1. 

4.2.2  Maze 

The  maze  was  created  using  the  MAGIC  Toolkit.  It  was  composed  of  rectangular  prisms 
and  cylinders  such  that  when  one  looks  at  the  projection  in  the  x-z  plane  the  maze  can 
be  seen.  It  had  two  additional  objects  in  the  scene  acting  as  a  starting  block  and  finishing 
block  that  triggered  the  timer  and  the  log  of  locations  and  errors.  The  starting  block  was 
located  at  the  center  of  the  workspace,  and  the  finishing  block  was  located  in  the  lower  half 
of  the  workspace. 

The  maze  started  with  the  cursor  at  the  center  of  the  visual  and  haptic  workspace. 
Figure  4-2  shows  the  x-z  projection  of  the  maze.  The  user  had  to  move  slightly  to  the  left 
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Figure  4-1:  Experimental  Apparatus 

to  contact  the  green  starting  block  so  as  to  trigger  the  timer.  Touching  the  starting  block 
resulted  in  a  change  of  color  from  green  to  light  green,  thus  giving  the  subject  a  visual  cue 
that  the  trial  began.  Once  the  subject  released  the  starting  block,  the  timer  started.  The 
recording  of  the  3-dimensional  location  of  the  cursor  started  as  soon  as  the  subject  was 
presented  with  the  maze  and  ended  when  the  subject  touched  the  finishing  block. 

The  subject  followed  the  path  of  the  maze  to  a  black  finishing  block.  Once  the  subject 
touched  the  finishing  block,  the  timer  and  log  stopped  and  the  subject  was  asked  to  rate 
the  ease  of  completing  the  maze.  Then  the  next  trial  began. 


4.3  Visual-Haptic  Size  Variations 

Experiments  1,  2,  3,  and  4  used  the  four  visual-haptic  size  variations  listed  in  Table  4.1. 
There  were  two  sizes,  small  and  large,  for  the  visual  and  haptic  workspace.  The  small  size 
had  dimension  of  4”x2.25”.  The  large  size  was  1.5  times  larger  with  dimensions  of  6”x3.25”. 
A  scaling  factor  of  1.5  was  chosen  due  to  the  limits  of  the  haptic  workspace.  Variation  1 
had  a  large  visual  and  a  large  haptic  workspace.  Variation  2  had  a  large  haptic  workspace 
and  a  small  visual  workspace.  Variation  3  had  a  small  haptic  workspace  and  large  visual 
workspace.  Variation  4  had  a  small  visual  and  a  small  haptic  workspace.  Figure  4-3  shows 
a  graphical  representation  of  the  four  variations. 
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Figure  4-2:  A  Typical  Maze 


Table  4.1:  Visual-Haptic  Size  Variations 


Haptic 

Size 

Visual  Size 

Large  Small 

Large 

Small 

Variation  1  Variation  2 
Variation  3  Variation  4 
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Variation  1 


Haptics 

Graphics 

1.5:1 

Variation  2 


Variation  3  Variation  4 

Figure  4-3:  Variations  of  Two  Visual  Sizes  and  Two  Haptic  Sizes 

4.3.1  Experiment  1:  Tests  Accuracy 

In  Experiment  1,  the  subjects  were  told  to  complete  each  of  the  mazes  while  minimizing 
contact  with  any  wall  of  the  maze.  The  session  consisted  of  62  trials,  starting  with  10  train¬ 
ing  trials  of  Variation  1,  followed  by  52  randomly  distributed  trials  of  the  four  variations. 
The  last  28  of  the  62  trials  had  ratings  for  ease  of  use.  Seven  subjects  were  tested  in  this 
experiment. 

4.3.2  Experiment  2:  Tests  Speed,  With  Visual  and  Haptic  Guidance 

The  objective  in  Experiment  2  was  to  complete  the  maze  as  quickly  as  possible  by  seeing 
the  visual  position  of  the  cursor  and  feeling  the  haptic  cursor  against  the  surface  of  the 
maze  wall. 

The  session  consisted  of  45  trials.  There  were  five  training  trials  on  Variation  1  and  40 
random  trials  of  the  four  variations.  Since  the  five  subjects  in  this  experiment  were  also 
subjects  in  Experiment  1,  only  a  small  number  of  training  trials  were  needed  to  refresh  their 
memory  on  how  to  use  the  apparatus. 
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4.3.3  Experiment  3:  Tests  Speed,  Without  Visual  Cursor  Guidance 

The  objective  in  Experiment  3  was  to  complete  the  maze  as  quickly  as  possible.  However, 
unlike  Experiment  2,  the  monitor  displayed  only  the  maze,  but  there  was  no  visual  cursor 
to  inform  the  subject  of  the  current  location.  This  forced  each  subject  to  rely  exclusively 
on  his/her  haptic  sense  to  assess  the  location  of  the  endpoint  of  the  stylus. 

This  session  consisted  of  45  trials.  There  were  five  training  trials  on  Variation  1  and  40 
randomly  distributed  trials  of  the  four  variations.  The  same  five  subjects  who  performed 
in  Experiment  1  and  2  also  participated  in  this  experiment. 

4.3.4  Experiment  4:  Tests  Speed,  Without  Haptic  Guidance 

The  objective  in  Experiment  4  was  again  to  complete  the  maze  as  quickly  as  possible.  How¬ 
ever,  in  this  experiment,  there  was  no  haptic  feedback  when  the  subject  had  the  endpoint 
of  the  stylus  in  the  physical  space  of  the  walls  of  the  maze.  This  forced  each  subject  to  rely 
exclusively  on  his/her  visual  sense. 

This  session  consisted  of  45  trials.  There  were  five  training  trials  on  variation  1  and 
40  randomly  distributed  trials  of  the  four  variations.  The  five  subjects  who  participated  in 
this  experiment  also  performed  in  Experiment  1,  2,  and  3. 


4.4  Visual  Scaling  Effects  on  Training 

Experiments  5  and  6  used  the  visual- haptic  variations  listed  in  Table  4-2.  There  were 
four  visual  sizes  and  one  haptic  size.  Size  “Small”  had  a  visual  display  of  4”x2.25”.  Size 
“Medium”  was  1.5  times  larger  than  “Small”.  Size  “Large”  was  2  times  larger  than  “Small”. 
Size  “Ex-large”  was  2.5  times  larger  than  “Small” .  The  haptic  size  presented  was  the  same 
as  “Small”.  Variation  5  had  “Small”  visual  and  haptic  workspaces.  Variation  6  had  a 
“Medium”  visual  workspace  and  a  “Small”  haptic  workspace.  Variation  7  had  a  “Large” 
visual  workspace  and  a  “Small”  haptic  workspace.  Variation  8  had  an  “Ex-large”  visual 
workspace  and  a  “Small”  haptic  workspace.  Figure  4-4  shows  a  graphical  representation  of 
the  four  variations  used  in  the  visual  scaling  experiments. 

These  experiments  were  designed  to  investigate  the  performance  of  users  when  presented 
with  one  haptic  size  and  several  visual  sizes.  It  studied  the  effects  of  training  time  on  a 
small  visual  size  and  the  effects  of  training  on  an  extremely  large  visual  size  while  the  haptic 
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Table  4.2:  Visual  Scaling  Variations 


Variation  7  Variation  8 


Figure  4-4:  Variations  of  Four  Visual  Sizes  and  One  Haptic  Size 
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size  remained  the  same. 


4.4.1  Experiment  5:  Increasing  Visual  Scale  (Training  on  a  Small  Visual 
Size) 

The  subjects  were  asked  to  complete  each  of  the  mazes  while  minimizing  contact  with  any 
wall  of  the  maze.  The  session  consisted  of  62  trials.  Ten  were  training  trials  of  Variation  5. 
The  subjects  were  trained  on  a  “Small”  visual  workspace  and  a  “Small”  haptic  workspace 
then  presented  with  52  trials  with  a  random  distribution  of  the  four  variations.  The  last  28 
of  the  62  trials  had  ratings  for  ease  of  use.  There  were  seven  subjects. 

4.4.2  Experiment  6:  Decreasing  Visual  Scale  (Training  on  a  Ex-Large 
Visual  Size) 

The  subjects  were  asked  to  complete  each  of  the  mazes  while  minimizing  contact  with  any  of 
the  walls  of  the  maze.  The  session  consisted  of  62  trials.  In  this  experiment,  the  10  training 
trials  were  of  Variation  8.  The  subjects  were  trained  on  the  “Ex-large”  visual  workspace  and 
the  “Small”  haptic  workspace.  Training  on  a  large  visual  display  conveyed  the  impression 
that  everything  else  was  scaled  down.  The  remaining  52  trials  were  a  random  distribution 
of  the  four  variations.  The  last  28  of  the  62  trials  had  ratings  for  ease  of  use.  Seven  subjects 
participated  in  this  experiment. 


4.5  Cursor  Paradigms 

In  this  experiment,  the  haptic  size  and  the  visual  were  both  4”x2.25”.  There  were  however, 
four  different  cursor  paradigms.  As  described  in  Chapter  3,  the  four  paradigms  include: 
“mouse”,  “magnifying  lens”,  “tele- video” ,  and  “RC  car”.  Table  4-3  describes  the  different 
attributes  associated  with  each  of  these  paradigms. 

4.5.1  Experiment  7:  Position  and  Force  Control  Cursor  Paradigms 

The  objective  of  this  experiment  was  to  complete  the  maze  a  quickly  as  possible  without 
passing  though  the  walls  of  the  maze.  Due  to  the  limits  of  the  motor  torque  output,  the 
walls  of  the  maze  did  not  feel  infinitely  stiff.  It  was  therefore  possible  to  pass  through  the 
walls  when  too  much  force  was  applied,  such  trials  were  rejected  for  the  final  data  analysis. 
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Table  4.3:  Cursor  Paradigms 


Analogy 

“mouse” 

“magnifying 

lens” 

“tele¬ 

video” 

“RC  car” 

Paradigm 

1 

2 

3 

4 

Visual  Cursor 

moving 

static 

static 

moving 

Visual  Environment 

static 

moving 

BUBEEl 

static 

Haptic  Cursor 

moving 

moving 

static 

static 

Haptic  Environment 

static 

iiiuv'SSfll 

Control  Type 

Position 

Control 

Position 

Control 

Force 

Control 

Force 

Control 

There  were  seven  subjects  for  this  experiment.  There  were  72  trials  in  this  session.  This 
included  3  of  each  of  the  four  paradigms  for  training  for  a  total  of  12  training  trials,  48 
randomly  distributed  trials  of  the  four  paradigms,  and  12  randomly  distributed  trials  of  the 
four  paradigms  with  a  new  maze.  The  new  maze  was  constructed  using  the  same  number 
of  objects  with  a  slightly  different  arrangement.  The  size  and  difficulty  of  the  two  mazes 
were  approximately  the  same.  Figure  4-5  shows  the  x-z  display  of  the  new  maze. 


Figure  4-5:  New  Maze  for  Experiment  7 
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Chapter  5 


Results 


5.1  Performance  Measures 

5.1.1  Time  Performance 

The  time  at  which  a  subject  started  the  maze  by  touching  the  starting  block  at  the  time 
the  subject  touched  the  finishing  block  was  recorded.  The  mean  time  required  to  complete 
the  mazes  for  each  of  the  paradigms  or  variations  was  compared. 


5.1.2  Error  Performance 

There  were  two  error  measures,  wall  contact  errors  and  wall  crossing  errors.  For  experiments 
in  which  the  subjects  were  told  not  to  touch  the  surface  of  the  maze,  any  contact  was  counted 
as  a  wall  contact  error.  This  error  count  number  was  based  on  the  number  of  cycles  of  the 
servoloop  in  which  the  subject  was  in  contact  with  the  maze  wall.  This  was  basically  a  sum 
of  the  time  in  which  the  subject  contacted  the  wall.  The  servoloop  ran  at  approximately 
500  Hz. 

In  all  the  experiments,  each  time  a  subject  used  too  much  force  and  passed  completely 
through  a  wall  of  the  maze,  it  was  counted  as  a  wall  crossing  error.  The  location  and 
time  was  recorded  for  each  subject’s  session.  There  was  a  viewing  program  for  each  of  the 
experiments  to  see  the  visual  display  of  the  subject’s  path  through  the  maze.  Using  this 
program,  a  manual  count  of  any  passage  through  the  walls  of  the  maze  could  be  determined. 


53 


5.1.3  Preference  Rating 


For  each  of  the  preference  rated  trials,  subjects  were  asked  to  rate  the  ease  of  completing 
the  trial  on  a  scale  of  1  to  5.  The  paradigm  or  variation  associated  with  that  trial  was  given 
points  equal  to  the  rating.  All  the  points  ■were  then  added  up  for  each  of  the  variations  or 
paradigms.  The  one  with  the  most  points  was  ranked  the  easiest  and  the  one  with  the  least 
amount  of  points  was  ranked  the  hardest. 

5.1.4  Performance  Ranking 

The  performance  ranking  gave  the  order  of  best  performance  to  worst  performance  by 
variation  or  paradigm  number.  For  Example,  if  variation  3  had  the  best  time  performance, 
followed  by  variations  4,  1,  and  2,  the  ranking  would  be  3,  4,  1,  2.  For  time  performance,  the 
least  amount  of  time  required  was  considered  the  best  and  the  most  amount  of  time  required 
was  considered  the  worst.  For  error  performance,  the  least  amount  of  error  was  considered 
the  best  and  the  most  amount  of  error  was  considered  the  worst.  For  preference  rating,  the 
highest  point  value  was  considered  the  best  and  the  lowest  point  value  was  considered  the 
worst. 


5.2  Methods  of  Analysis 

5.2.1  Statistical 

The  mean,  the  standard  error  of  the  mean,  and  the  median  of  the  results  were  calculated 
for  each  of  the  variations  of  the  experiments.  In  the  experiments  in  which  the  subjects  were 
asked  to  minimize  error,  subjects  were  able  to  complete  trials  without  touching  the  walls 
of  the  maze.  The  number  of  trials  in  which  the  subjects  had  no  errors  were  added  for  each 
variation  in  the  accuracy  experiments. 

5.2.2  Boxplot 

A  boxplot  as  in  Figure  5-1,  shows  the  important  statistical  characteristics  about  a  group 
of  data.  The  box  is  centered  at  the  median  value  and  shows  the  upper  quartile  and  lower 
quartile  of  the  data.  The  notch  represents  a  robust  estimate  of  the  uncertainty  about  the 
median  for  a  boxplot  to  boxplot  comparison.  Whiskers  are  lines  extending  from  each  end 
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of  the  box  to  show  the  extent  of  the  data.  The  whiskers  are  a  function  of  the  interquartile 
range.  The  interquartile  range  is  a  robust  estimate  of  the  spread  of  the  data  since  changes  in 
the  upper  and  lower  25%  do  not  affect  it.  It  is  calculated  by  subtracting  the  25th  percentile 
from  the  75th  percentile  and  multiplying  by  1.5. 

5.3  Visual-Haptic  Size  Variations 

Table  5.1  shows  the  matching  of  two  visual  sizes  to  two  haptic  sizes  for  the  different  varia¬ 
tions  used  in  Experiments  1,  2,  3,  and  4.  This  set  of  experiments  tested  the  significance  of 
changing  both  the  visual  and  haptic  size  parameters. 


Table  5.1:  Visual-Haptic  Size  Variation 


Haptic 

Size 

Visual  Size 

Large  Small 

Large 

Small 

Variation  1  Variation  2 
Variation  3  Variation  4 

5.3.1  Experiment  1:  Tests  Accuracy 

In  this  experiment,  the  subjects  were  told  to  complete  the  maze  as  accurately  as  possible. 
They  were  asked  to  minimize  contact  with  the  walls  of  the  maze  as  they  followed  the  path 
from  the  starting  block  to  the  finishing  block. 

Table  5-2  show's  the  mean  time  results  for  the  seven  subjects  of  Experiment  1.  Subject 
5  required  much  more  time  to  complete  the  maze  than  the  rest  of  the  subjects.  A  boxplot 
of  the  time  results  for  all  seven  subjects,  displayed  in  Appendix  B,  shows  that  most  of  the 
data  for  subject  5  was  outside  the  range  of  data.  Another  set  of  statistics  were  calculated 
without  the  data  from  subject  5.  The  boxplot  of  this  is  shown  in  Figure  5-1.  The  trends 
were  the  same  for  either  case,  but  the  standard  deviation  was  less  for  the  6  subjects.  The 
following  discussion  is  based  on  the  data  from  6  subjects. 

Variation  3,  with  small  haptics  and  large  graphics,  was  completed  in  the  least  amount  of 
time  for  six  out  of  the  seven  subjects.  The  average  time  required  to  complete  the  maze  with 
this  variation  was  15.0  ±  0.5  seconds.  Variation  4,  with  small  haptics  and  small  graphics, 
was  the  second  fastest  in  overall  time  performance  with  a  mean  time  of  15.9  ±  0.5  seconds. 
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Table  5.2:  Mean  Time  Performance  for  Experiment  1:  Accuracy 


!■■■■ 

Var  1  (sec) 

Var  2  (sec) 

Var  3  (sec) 

Var  4  (sec) 

ranking 

Subject  1 

17.1 

17.5 

13.9 

14.7 

3, 4, 1,2 

Subject  2 

17.9 

19 

16.1 

16.3 

3, 4, 1,2 

23.5 

24.7 

H 

wm&m 

Ei&mi 

Subject  4 

15.1 

15.6 

13.5 

13.8 

WSM 

Subject  5  * 

46.2 

50.7 

37.5 

41 

3, 4, 1,2 

ncBumii 

11.1 

12.2 

8.6 

9.2 

3, 4, 1,2 

Subject  7 

19.8 

22.9 

16.8 

20.4 

3, 1,4, 2 

Statistics  with  7  Subjects 

Average 

21.5  ±  1.3 

23.2  ±  1.5 

19.5  ±  1.1 

3, 4, 1,2 

Median 

17.4 

19.7 

15.9 

16.3 

3,4, 1,2 

Statistics  with  Subjects  1,2, 3, 4, 6, 7 

Average  (6) 

15.0  ±  0.5 

15.9  ±  0.5 

3, 4, 1,2 

Median  (6) 

16.8 

19.0 

14.6 

15.9 

3,4, 1,2 

Boxplot  for  Time  Results  of  Experiment  1 :  Accuracy  (6  Subjects) 


Variation 


Figure  5-1:  Boxplot  of  Time  Results  for  Experiment  1:  Accuracy.  Variation  1:  mean  = 
17.4  ±  0.5,  median  =  16.8.  Variation  2:  mean  =  18.7  ±  0.6,  median  =  19.0.  Variation  3: 
mean  =  15.0  ±  0.5,  median  =  14.6.  Variation  4:  mean  =  15.9  ±  0.5,  median  =  15.9.  (sec) 
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Both  of  these  variations  had  small  haptics  and  required  12%  less  time  to  complete  than  the 
variations  with  large  haptics.  An  explanation  is  that  since  the  physical  path  to  complete  the 
smaller  haptic  maze  was  shorter,  it  required  less  time  than  the  one  with  a  longer  path.  The 
two  variations  with  large  haptics  required  the  most  amount  of  time  to  complete.  Variation 
1,  with  large  graphics  and  large  haptics,  was  completed  faster  than  Variation  2,  with  small 
graphics  and  large  haptic.  Variation  1  required  17.4  ±  0.5  seconds  to  complete  on  average. 
Variation  2  required  18.7  ±  0.6  seconds  to  complete  on  average.  These  results  showed  that 
subjects  performed  faster  when  they  were  given  small  haptics  than  when  they  were  given 
large  haptics.  The  results  also  showred  that  if  subjects  were  presented  with  single  haptic 
size  two  visual  sizes,  performance  was  better  with  the  large  visual  size  than  with  the  small 
visual  size.  The  median  time  results  were  consistent  with  the  mean  time  results.  The 
median  times  required  were  14.6,  15.9,  16.8,  and  19.0  seconds  for  Variations  3,  4,  1,  and  2 
respectively. 

Table  5-3  lists  the  mean  error  results  for  the  seven  subjects  of  Experiment  1.  The 
following  discussion  is  based  on  the  analysis  of  the  data  from  six  subjects.  Overall,  the 
error  count  differences  among  the  four  variations  were  much  smaller  than  the  completion 
times.  Relative  to  the  high  standard  error  of  the  mean  in  the  error  count,  the  differences 
in  the  mean  values  were  negligible.  Variation  1,  with  large  haptics  and  large  graphics,  had 
the  lowest  error  count,  451  ±  55.  Subjects  had  more  space  visually  and  haptically  to  move, 
therefore,  it  was  reasonable  to  see  such  a  result.  Variation  4,  with  small  graphics  and  small 
haptics,  also  had  a  low  error  count,  465  ±  59.  One  explanation  for  this  might  be  that  small 
graphics  instilled  a  sense  of  caution  in  the  subjects  resulting  in  a  smaller  error  count.  Both 
of  these  variations  had  a  relatively  low  error  count  and  had  a  one  to  one  visual  to  haptic 
ratio.  A  high  correlation  between  the  visual  and  haptic  workspace  resulted  in  a  low  error 
count. 

Variations  2  and  3,  both  of  which  had  a  skewed  size  correlation  between  the  visual  and 
haptic  workspace,  had  high  error  counts.  The  small  graphics  of  Variation  2  made  subjects 
cautious,  while  the  large  haptics  gave  subjects  more  space  to  move,  resulting  in  relatively 
fewer  errors,  498  ±  58.  Variation  3,  with  small  haptic  and  large  graphics,  had  the  highest 
error  count,  543  ±  61.  This  variation  had  large  graphics  which  led  the  subjects  to  relax 
their  caution  in  not  touching  the  walls  resulting  in  larger  errors.  Variation  3,  which  also 
had  small  haptics,  required  a  high  degree  of  motor  control  to  maneuver  in  a  small  haptic 
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Table  5.3:  Mean  Error  Counts  for  Experiment  1:  Accuracy 


Var  1 

Var  2 

Var  3 

Var  4 

ranking 

Subject  1 

627 

644 

708 

603 

4, 1,2, 3 

Subject  2 

160 

184 

198 

113 

4, 1,2, 3 

Subject  3 

51 

88 

105 

85 

1,4, 2, 3 

Subject  4 

309 

384 

542 

424 

1,2, 4, 3 

Subject  5  * 

13 

130 

30 

103 

1,3, 4, 2 

Subject  6 

1292 

1376 

1409 

1209 

4, 1,2, 3 

Subject  7 

268 

311 

301 

360 

1,3, 2, 4 

Statistics  with  7  Subjects 

Average 

389  ±  50 

445  ±  54 

470  ±  56 

414  ±  52 

1,4, 2, 3 

Median 

209 

227 

302 

230 

1,2, 4, 3 

no  error  trials 

18 

11 

17 

14 

1,3, 4, 2 

Statistics  with  Subjects  1,2, 3, 4, 6, 7 

Average(6) 

451  ±  55 

498  ±  58 

543  ±  61 

465  ±  59 

1,4, 2, 3 

Median  (6) 

280 

295 

365 

312 

1,2, 4, 3 

no  error  trials 

8 

3 

8 

8 

(1,3,4),  2 

space. 

The  median  errors  showed  a  slightly  different  trend  with  error  counts  at  280,  295,  312, 
and  356  for  Variations  1,  2,  4,  and  3  respectively.  There  was  a  number  of  trials  in  which 
the  subjects  made  no  errors.  For  Variations  1,  3,  4,  and  2,  there  were  8,  8,  8,  and  3  trials  in 
which  the  subjects  had  no  errors  when  completing  the  maze.  Figure  5-2  shows  the  boxplot 
of  the  error  results  for  Experiment  1. 

There  were  no  wall  crossing  errors  any  of  the  variations,  since  the  subjects  were  told  to 
minimize  contact  with  the  walls. 

Table  5-4  shows  the  preference  rankings  of  the  variations  for  the  seven  subjects  in  Ex¬ 
periment  1.  A  high  resultant  score  was  equivalent  to  a  strong  perception  of  ease.  According 
to  the  rankings,  Variation  3,  with  small  haptics  and  large  graphics,  was  perceived  to  be  the 
easiest  to  complete.  Several  subjects  also  equally  rated  Variation  1  and  3,  both  of  which 
had  large  graphics.  This  implied  that  regardless  of  the  size  of  the  haptic  workspace,  so  long 
as  the  visual  presentation  of  that  workspace  was  large,  the  user  felt  more  at  ease.  Many 
subjects  rated  Variations  2  and  4  as  equally  hard  to  complete.  These  variations  both  had 
small  graphics.  Once  again,  the  effects  of  visual  presentation  dominated  performance.  Vari¬ 
ation  1  was  ranked  second  easiest,  followed  by  Variation  4.  Variation  2  with  large  haptics 
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Boxplot  for  Error  Counts  of  Experiment  1 :  Accuracy  (6  Subjects) 


Figure  5-2:  Boxplot  of  Error  Results  for  Experiment  1:  Accuracy.  Variation  1:  mean  =  451 
±  55,  median  =  280.  Variation  2:  mean  =  498  ±  58,  median  =  295.  Variation  3:  mean  = 
543  ±  61,  median  =  365.  Variation  4:  mean  =  465  ±  59,  median  =  312.  (counts) 
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and  small  graphics  was  rated  the  hardest  to  complete. 


Table  5.4:  Preference  Rankings  for  Experiment  1:  Accuracy 


Sub  1 

Sub  2 

Sub  3 

Sub  4 

Sub  5 

Sub  6 

Sub  7 

total 

Var  1 

32 

28 

28 

25 

28 

23 

33 

197 

Var  2 

29 

16 

21 

23 

19 

13 

21 

142 

Var  3 

34 

26 

29 

33 

34 

215 

Var  4 

15 

22 

23 

19 

27 

21 

160 

Pref. 

3, 4, 1,2 

3, 1,2,4 

1,3, 4, 2 

3, 1,2, 4 

3, 1,2,4 

In  summary,  there  was  an  inverse  relationship  between  time  performance  and  error 
performance;  the  faster  the  time,  the  higher  the  error.  Variations  with  smaller  haptics  were 
faster  to  complete,  possibly  because  the  physical  length  of  the  path  was  less  than  that  for 
the  larger  haptics.  Graphics,  when  large,  also  allowed  subjects  to  easily  navigate  through 
the  maze.  There  was  a  perception  of  more  space  and  fewer  constraints  when  subjects  were 
presented  with  large  graphics. 

5.3.2  Experiment  2:  Tests  Speed,  With  Visual  and  Haptic  Guidance 

In  this  experiment,  the  subjects  were  told  to  minimize  the  time  to  completion  and  to  use  the 
walls  of  the  maze  as  a  guide  to  complete  the  maze.  Five  training  trials  were  given  followed 
by  40  random  trials  of  the  four  variations. 

Table  5-5  shows  the  mean  time  performance  for  the  five  subjects  in  Experiment  2. 
Subject  3  required  much  more  time  to  complete  the  mazes  than  any  of  the  other  subjects  for 
experiments  2,  3,  and  4.  This  subject  was  the  subject  who  had  outlayer  data  in  Experiment 
1.  The  boxplots  of  the  results  with  all  five  subjects  can  be  seen  in  Appendix  B.  The 
following  discussion  is  based  on  the  statistics  calculated  with  subjects  1,  2,  4,  and  5.  The 
trends  among  the  variations  were  the  same  for  both  four  subjects  and  five  subjects,  though 
the  variance  in  the  data  was  less  for  four  subjects. 

Variation  3,  with  small  haptics  and  large  graphics,  had  the  fastest  time  performance, 
6.0  ±  0.2  seconds.  Variation  4,  with  small  haptics  and  small  graphics,  followed  closely  with 
an  average  time  performance  of  6.1  ±  0.2  seconds.  Figure  5-3  shows  the  boxplot  of  the  time 
results. 

Variation  1,  with  large  haptics  and  large  graphics  had  the  next  fastest  completion  time, 
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Table  5.5:  Mean  Time  Performance  for  Experiment  2:  Speed,  Visual  and  Haptic  Feedback 


Var  1  (sec) 

Var  2  (sec) 

Var  3  (sec) 

Var  4  (sec) 

pref. 

imsmmm 

6.3 

6.7 

5.4 

5.5 

6 

6.1 

5.3 

5.1 

4, 3, 1,2 

18.7 

20.2 

16.2 

16.5 

3, 4, 1,2 

ISHIRffiRFSMi 

8.6 

9 

7.6 

8.3 

EESai 

Subject  5 

6.7 

7 

5.7 

5.5 

isaai 

Statistics  with 

5  Subjects 

IKSSSES5I881 

9.3  ±  0.7 

9.8  ±  0.8 

8.0  ±  0.6 

8.2  ±  0.6 

Median 

7.1 

7.3 

6 

5.8 

4, 3, 1,2 

Statistics  with  Subjects  1,2, 4, 5 

Average  (4) 

6.9  ±  0.2 

7.2  ±  0.2 

6.0  ±  0.2 

6.1  ±  0.2 

3, 4, 1,2 

Median  (4) 

6.9 

7.2 

5.7 

5.6 

4, 3, 1,2 

Boxplot  of  Time  Results  for  Experiment  2:  Visual  and  Haptic  Feedback  (4  Subjects) 


Figure  5-3:  Boxplot  of  the  time  results  for  Experiment  2:  Speed,  Visual  and  Haptic  Feed¬ 
back.  Variation  1:  mean  =  6.9  ±  0.2,  median  =  6.9.  Variation  2:  mean  =  7.2  ±  0.2, 
median  =  7.2.  Variation  3:  mean  =  6.0  ±  0.2,  median  =  5.7.  Variation  4:  mean  =  6.1  ± 
0.2,  median  =  5.6.  (sec) 
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6.9  ±  0.2  seconds.  Variation  2  had  the  slowest  completion  time,  7.2  ±  0.2  seconds.  Based 
on  the  median  times,  the  variations  were  ranked  as  4,  3,  1,  and  2  with  times  of  5.6,  5.7,  6.9 
and  7.2  seconds  respectively. 

Since  the  walls  of  the  maze  were  not  infinitely  stiff,  subjects  were  able  to  pass  through 
the  walls  if  too  much  force  was  applied.  There  was  a  total  of  zero  wall  crossing  errors  for 
Variation  1,  two  for  Variation  2,  eight  for  Variation  3,  and  four  for  Variation  4.  Variation 
3,  with  small  haptics  and  large  graphics,  had  a  large  number  of  wall  crossing  errors.  This 
can  be  explained  as  an  effect  of  visual  dominance.  When  subjects  touched  the  walls  of  the 
maze,  the  wall  felt  less  stiff  than  if  the  haptics  and  graphics  were  the  same  size.  Since  the 
graphics  were  larger  than  the  haptics,  the  subjects  thought  they  had  more  room  to  move 
through  than  they  actually  did. 

Table  5-6  shows  the  preference  rankings  for  the  variations  in  Experiment  2.  Variation  4, 
with  small  haptics  and  small  graphics,  was  rated  easiest  to  use  by  three  out  of  five  subjects. 
One  subject  rated  Variations  1  and  3  equally  as  the  easiest  to  use,  indicating  a  preference 
for  large  graphics.  Subject  5  rated  Variations  1,  3,  and  4  as  equally  easy  to  use.  In  general, 
Variation  2  with  large  haptics  and  small  graphics  was  rated  the  most  difficult  to  use. 

Table  5.6:  Preference  Rankings  for  Experiment  2:  Speed,  Visual  and  Haptic  Feedback 


Var 

Subject  1 

Subject  2 

Subject  3 

Subject  4 

Subject  5 

Total 

■Blf 

34 

33 

46 

48 

23 

mm 

2 

34 

30 

40 

41 

15 

160 

mm 

37 

47 

48 

29 

191 

wm 

37 

40 

45 

44 

30 

196 

Pref 

4,3,2, 1 

4, 3, 1,2 

3, 1,4,2 

(13), 4, 2 

4, 3, 1,2 

4,3, 1,2 

5.3.3  Experiment  3:  Tests  Speed,  Without  Visual  Cursor  Guidance 

In  Experiment  3,  subjects  were  told  to  minimize  the  time  to  completion  and  to  use  the  walls 
of  the  maze  as  a  guide.  They  were  presented  with  a  visual  display  of  the  maze  without  the 
visual  cursor  update.  This  forced  the  subjects  to  rely  on  their  haptic  sense.  Table  5-7  shows 
the  mean  time  performance  results  for  Experiment  3. 

Variation  4,  with  small  haptics  and  small  graphics,  was  completed  the  fastest  on  average, 
7.4  ±  0.4  seconds.  Variation  3,  with  small  haptics  and  large  graphics,  was  only  slightly 
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Table  5.7:  Mean  Time  Performance  Times  for  Experiment  3:  Speed,  Haptic  Feedback  Only 


Var  1  (sec) 

Var  2  (sec) 

Var  3  (sec) 

Var  4  (sec) 

ranking 

Subject  1 

7.4 

9.2 

7.4 

5.8 

IKDQEBI 

Subject  2 

7.4 

10.4 

8.2 

6 

EJU&ai 

Subject  3 

17.8 

20.9 

12.9 

13.3 

FEW-Ml 

Subject  4 

11.6 

11.8 

9.4 

11.1 

ElilKflll 

Subject  5 

6.9 

7.9 

5.4 

7 

3, 1,4, 2 

Statistics  with  5  Subjects 

Average 

10.2  ±  0.7 

12  ±  0.9 

8.6  ±  0.4 

8.6  ±  0.5 

4, 3, 1,2 

Median 

8.2 

10.2 

8.3 

7.6 

4, 1,3, 2 

Statistics  with  Subjects  1,2, 4, 5 

Average  (4) 

8.3  ±  0.4 

9.7  ±  0.5 

7.6  ±  0.4 

7.4  ±  0.4 

4, 3, 1,2 

Median  (4) 

7.7 

9.3 

7.6 

6.7 

4, 3, 1,2 

behind  with  a  time  of  7.6  ±  0.4  seconds  followed  by  Variation  1,  with  large  haptics  and 
large  graphics,  at  a  time  of  8.3  ±  0.4  seconds.  Variation  2,  with  large  haptics  and  small 
graphics,  required  the  most  time  to  complete,  9.7  ±  0.5  seconds.  The  median  times  were 
in  the  same  performance  order  as  the  mean  times  with  6.7,  7.6,  7.7,  and  9.3  seconds  for 
Variations  4,  3,  1,  and  2  respectively.  Figure  5-4  shows  a  boxplot  of  the  time  results  for 
Experiment  3. 

Since  the  walls  of  the  maze  were  not  infinitely  stiff,  subjects  were  able  to  pass  through 
the  walls  if  too  much  force  was  applied.  There  was  a  total  of  3  wall  crossing  errors  for 
Variation  1,  6  for  Variation  2,  17  for  Variation  3,  and  7  for  Variation  4.  The  wall  crossing 
errors  were  very  high  for  Variations  3.  As  explained  earlier  for  Experiment  2,  this  was 
probably  an  effect  of  visual  dominance.  In  general,  the  wall  crossing  errors  were  higher 
for  this  experiment  than  for  Experiment  2.  This  was  because  this  experiment  required  the 
subjects  to  stay  in  contact  with  the  wall  of  the  maze.  The  longer  they  stayed  in  contact 
with  the  wall,  the  more  likely  they  were  to  make  a  wall  crossing  error. 

Table  5-8  shows  the  preference  rankings  for  the  variations  in  Experiment  3.  Variation 
1,  with  large  haptics  and  large  graphics,  was  rated  easiest  by  a  wide  margin.  Variation  4 
with  small  graphics  and  small  haptics  was  then  rated  next  easiest.  Variation  2  was  followed 
by  Variation  3  for  ease  of  use. 
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Boxplot  of  Time  Results  for  Experiment  3:  Haptic  Feedback  Only  (4  Subjects) 


Figure  5-4:  Boxplot  of  the  time  results  for  Experiment  3:  Speed,  Haptic  Feedback  Only. 
Variation  1:  mean  =  8.3  ±  0.4,  median  =  7.7.  Variation  2:  mean  =  9.7  ±  0.5,  median  = 
9.3.  Variation  3:  mean  =  7.6  ±  0.4,  median  =  7.6.  Variation  4:  mean  =  7.4  ±  0.4,  median 
=  6.7.  (sec) 


Table  5.8:  Preference  Rankings  for  Experiment  3:  Haptic  Feedback  Only 


Subject  1 

Subject  2 

Subject  3 

Subject  4 

Subject  5 

Total 

Variation  1 

35 

26 

37 

42 

32 

172 

Variation  2 

30 

20 

34 

39 

29 

152 

Variation  3 

35 

21 

40 

33 

12 

141 

Variation  4 

23 

38 

33 

28 

158 

Preference 

(4(31))2 

1,4, 3, 2 

3, 4, 1,2 

KESHI 
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5.3.4  Experiment  4:  Tests  Speed,  Without  Haptic  Guidance 


In  Experiment  4,  subject  were  told  to  minimize  completion  time  and  to  stay  on  the  path 
of  the  maze.  They  were  presented  with  the  maze  and  the  cursor  visually  but  given  no 
haptic  feedback  when  they  touched  the  walls  of  the  maze.  Table  5-9  shows  the  mean  time 
performance  for  Experiment  4. 


Table  5.9:  Mean  Time  Performance  for  Experiment  4:  Speed,  Visual  Feedback  Only 


Var  1  (sec) 

Var  2  (sec) 

Var  3  (sec) 

Var  4  (sec) 

ranking 

Subject  1 

9.4 

10.6 

8.5 

8.7 

3, 4, 1,2 

Subject  2 

13.2 

14.1 

12.5 

12.3 

4, 3, 1,2 

msmm 

20.6 

18.8 

18.2 

4, 3, 1,2 

Subject  4 

12.7 

12.5 

10.7 

10.8 

EJESft*! 

Subject  5 

8.4 

9.2 

7.5 

7.2 

usai 

Statistics  with  5  Subjects 

Average 

11.6  ±  0.6 

11.5  ±  0.5 

4, 3, 1,2 

Median 

12.2 

12.5 

10.9 

10.8 

4, 3, 1,2 

Statistics  with  Subjects  1,2, 4, 5 

Average  (4) 

10.1  ±  0.4 

8.9  ±  0.3 

4,3, 1,2 

Median  (4) 

9.5 

10.6 

8.4 

8.7 

3,4, 1,2 

Subjects  completed  Variation  4,  with  the  small  haptics  and  small  graphics,  the  fastest 
with  an  average  time  of  8.9  ±  0.3  seconds.  There  was  no  significant  difference  in  the 
completion  time  between  Variation  4  and  Variation  3  (9.0  ±  0.3  seconds),  both  of  which  had 
a  small  haptic  workspace.  There  was  also  no  significant  difference  in  completion  times  for 
Variations  1  and  2,  with  times  of  10.1  ±  0.4  and  10.7  ±  0.3  seconds  respectively.  Variations 
1  and  2  required  10  to  20%  more  time  to  complete  than  Variations  3  and  4.  Comparing 
between  variations  with  a  large  haptic  size,  the  one  which  also  had  a  small  visual  size 
required  8-10%  more  time  than  the  one  with  a  large  visual  size.  The  ranking  based  on 
median  time  is  Variation  3,  4,  1,  and  2  with  completion  times  of  8.4,  8.7,  9.5,  and  10.6 
seconds.  Figure  5-5  shows  the  boxplot  of  the  completion  times  for  Experiment  4. 

There  were  no  wall  crossing  errors  for  Experiment  4.  Since  the  subjects  only  had  visual 
feedback,  and  they  were  told  to  stay  within  the  path  of  the  maze,  no  one  strayed  off  course 
to  make  a  wall  crossing  error. 

Table  5-10  shows  the  preference  rankings  for  the  variations  in  Experiment  4.  Subjects 
generally  preferred  large  visual  displays  to  small  visual  displays. 
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Boxplot  of  Time  Results  for  Experiment  4:  Visual  Feedback  Only  (4  Subjects) 


Figure  5-5:  Boxplot  of  the  time  results  for  Experiment  4:  Speed,  Visual  Feedback  Only. 
Variation  1:  mean  =  10.1  ±  0.4,  median  =  9.5.  Variation  2:  mean  =  10.7  ±  0.3,  median  = 
10.6.  Variation  3:  mean  =  9.0  ±  0.3,  median  =  8.4.  Variation  4:  mean  =  8.9  ±  0.3,  median 
=  8.7.  (sec) 


Table  5.10:  Preference  Rankings  for  Experiment  4:  Speed,  Visual  Feedback  Only 


Subject  1 

Subject  2 

Subject  3 

Subject  4 

Subject  5 

Total 

Variation  1 

32 

32 

40 

46 

27 

177 

Variation  2 

25 

19 

33 

42 

18 

137 

Variation  3 

27 

32 

40 

47 

34 

180 

Variation  4 

30 

17 

36 

41 

29 

153 

Pref 

1,4, 3, 2 

1,3, 2, 4 

unrif 

3, 1,2, 4 

3, 4, 1,2 

3, 1,4, 2 
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5.4  Visual  Scaling  Effects  on  Training 


In  the  visual  scaling  experiments,  the  mazes  all  had  the  same  haptic  workspace  size  but 
had  different  visual  workspace  sizes.  In  one  experiment,  the  subjects  were  trained  on  a 
small  visual  workspace.  In  the  other  experiment,  the  subjects  were  trained  on  a  large  visual 
workspace.  Table  5-11  lists  the  variations  used  in  this  set  of  experiments.  The  subjects  were 
told  to  minimize  contact  with  the  walls  in  both  visual  scaling  experiments.  There  were  a 
total  of  14  subjects,  seven  subjects  for  the  increasing  visual  scale  experiment,  and  seven 
different  subjects  for  the  decreasing  visual  scale  experiment. 


Table  5.11:  Visual  Scaling  Variations 


Visual-Haptic  Ratio 

Name 

Variation  5 

1:1 

Small 

Variation  6 

1.5:1 

Medium 

Variation  7 

2:1 

Large 

Variation  8 

2.5:1 

Ex-Large 

5.4.1  Experiment  5:  Increasing  Visual  Scale 

In  this  experiment,  the  subjects  were  trained  on  a  small  visual  workspace.  Table  5-12  shows 
the  mean  time  performance  results  for  seven  subjects. 


Table  5.12:  Mean  Time  Performance  for  Experiment  5:  Increasing  Visual  Scale 


Small 

Medium 

Large 

Ex-large 

Var  5  (sec) 

Var  6  (sec) 

Var  7  (sec) 

jEsmi 

Subject  1 

38 

39 

34 

36 

7, 8, 5, 6 

Subject  2 

14.4 

13.8 

14.1 

13.5 

8, 6, 7, 5 

Subject  3 

14.4 

12.6 

12.6 

12.4 

8, 7, 6, 5 

Subject  4 

12 

11.7 

10.9 

11.5 

KE823 

Subject  5 

35.4 

32.7 

27.7 

27.4 

8, 7, 6, 5 

Subject  6 

13.9 

13.5 

13.7 

13.9 

6, 7, 8, 5 

28.4 

26.4 

23.2 

22.7 

8, 7, 6, 5 

Average 

22.4  ±  1.2 

21.4  ±  1.2 

19.6  ±  1.0 

7, 8, 6, 5 

Median 

16.3 

14.2 

15.5 

15.4 

6, 8, 7, 5 

Variation  “Large”  had  the  fastest  mean  time  for  completion,  19.5  ±  0.9  seconds  .  Varia¬ 
tion  “Ex-large”  was  second  fastest  with  19.6  ±  1.0  seconds,  followed  by  Variation  “Medium” 
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with  21.4  ±  1.2  seconds.  Variation  “Small”  was  slowest  with  a  mean  completion  time  of 
22.4  ±  1.2  seconds.  The  mean  time  and  standard  deviations  showed  that  there  was  very 
little  difference  among  the  four  size  variations.  However,  the  median  values  of  the  variations 
showed  that  there  was  a  significant  improvement  in  performance  time  between  “Small”  and 
“Medium”,  which  had  completion  times  of  16.3  seconds  and  14.2  seconds  respectively,  and  a 
distinct  decrease  in  performance  as  the  visual  size  increased.  The  completion  time  dropped 
from  14.2  seconds  to  15.5  seconds  and  15.4  seconds  for  Variation  “Medium”  to  Variation 
“Large”  and  Variation  “Ex-large”  respectively.  Figure  5.6  shows  a  boxplot  of  the  time 
performance  for  the  increased  visual  scaling  experiment. 


Boxplot  of  Time  Results  for  Experiment  5:  Increased  Visual  Scaling 


Figure  5-6:  Boxplot  of  the  time  results  for  Experiment  5:  Increasing  Visual  Scale.  Variation 
5  (“Small”):  mean  =  22.4  ±  1.2,  median  =  16.3.  Variation  6  (“Medium”):  mean  =  21.4  ± 
1.2,  median  =  14.2.  Variation  7  (“Large”):  mean  =  19.5  ±  0.9,  median  =  15.5.  Variation 
8  (“Ex-large”):  mean  =  19.6  ±  1.0,  median  =  15.4.  (sec) 

Table  5-13  shows  the  mean  error  for  the  size  variations  presented  to  the  seven  subjects 
in  Experiment  5.  Variation  “Medium”  had  the  least  amount  of  error  on  average,  319  ± 
40.  Variation  “Ex-large”  had  the  next  lowest  amount  of  error  with  326  ±  46,  followed  by 
Variation  “Large”  with  335  ±  42.  Variation  “Small”  had  the  most  amount  of  error  with  406 
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±  47  counts.  Ordering  according  to  the  median  values  was  the  same:  Variation  “Medium” 
had  the  least  amount  of  error  followed  by  “Ex-large” ,  “Large” ,  and  “Small,”  with  of  error 
counts  of  143,  160,  180,  and  217  respectively.  There  were  several  trials  in  which  the  subjects 
had  no  errors.  Variation  “Small”  had  12  trials,  “Medium”  had  18  trials,  “Large”  had  19 
trials,  and  “Ex-large”  had  24  trials  with  no  errors.  Figure  5-7  shows  a  boxplot  of  the  error 
results  for  the  increased  visual  scaling  experiment. 

Table  5.13:  Mean  Error  Performance  for  Experiment  5:  Increasing  Visual  Scale 


Small 

Medium 

Large 

Ex-large 

Var  5 

Var  6 

Var  7 

EEH 

Subject  1 

155 

41 

21 

50 

Subject  2 

885 

693 

897 

888 

Egaai 

Subject  3 

330 

215 

153 

143 

8, 7, 6, 5 

Subject  4 

405 

338 

417 

mm 

Subject  5 

123 

67 

119 

IdHdail 

Subject  6 

901 

855 

llBHi 

666 

8, 7, 6, 5 

Subject  7 

45 

29 

29 

15 

8, 6, 7, 5 

Average 

406  ±  47 

319  ±  40 

335  ±  42 

326  ±  46 

— ■ 

Median 

217 

143 

180 

160 

no  error  trials 

12 

18 

19 

24 

8, 7, 6, 5 

There  were  no  wall  crossing  errors  in  this  experiment  since  the  subjects  were  told  to 
minimize  wall  contact. 

Table  5-14  shows  the  results  of  the  preference  rankings  of  the  variations  for  Experiment 
5.  Subjects  consistently  and  unanimously  ranked  Variation  “Ex-large”  as  the  easiest  to 
complete  and  Variation  “Small”  as  the  hardest  to  complete.  Most  subjects  ranked  strictly 
according  to  size.  However,  two  subjects  found  Variation  “Medium”  to  be  easier  than 
Variation  “Large”. 


Table  5.14:  Preference  Rankings  for  Experiment  5:  Increasing  Visual  Scale 


Var 

Sub  1 

Sub  2 

Sub  3 

Sub  4 

Sub  5 

Sub  6 

Sub  7 

Total 

5)  Small 

14 

28 

10 

25 

14 

22 

17 

130 

6)  Medium 

21 

31 

25 

32 

22 

25 

23 

179 

7)  Large 

28 

30 

26 

29 

27 

30 

28 

198 

8)  Ex-large 

29 

33 

31 

_ 

35 

32 

34 

32 

226 

8, 7, 6, 5 

8, 6, 7, 5 

8, 7, 6, 5 

8, 6, 7, 5 

8, 7, 6, 5 

8, 7, 6, 5 
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Boxplot  of  Error  Resutls  for  Experiment  5:  Increased  Visual  Scaling 


Figure  5-7:  Boxplot  of  the  error  results  for  Experiment  5:  Increasing  Visual  Scale.  Variation 
5  (“Small”):  mean  =  406  ±  47,  median  =  217.  Variation  6  (“Medium”):  mean  =  319  ± 
40,  median  =  143.  Variation  7  (“Large”):  mean  =  335  ±  42,  median  =  180.  Variation  8 
(“Ex-large”):  mean  =  326  db  46,  median  =  160.  (counts) 
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5.4.2  Experiment  6:  Decreasing  Visual  Scale 


In  this  experiment,  subjects  were  trained  on  the  “Ex- large”  visual  size  and  told  to  minimize 
contact  with  the  walls  of  the  maze.  Table  5-15  shows  the  mean  time  required  to  complete 
each  of  the  variations  for  Experiment  6.  Subject  7  required  much  more  time  to  complete 
the  mazes  than  the  rest  of  the  subjects.  The  boxplot  of  the  results  with  seven  subjects  is 
shown  in  Appendix  B.  Most  of  the  trials  for  subject  7  were  outside  the  range  of  data.  A 
new  analysis  was  made  without  subject  7.  The  following  discussion  was  based  on  the  results 
with  six  subjects. 


Table  5.15:  Mean  Time  Performance  for  Experiment  6:  Decreasing  Visual  Scale 


Small 

Medium 

Large 

Ex-large 

Var  5  (sec) 

Var  6  (sec) 

Var  7  (sec) 

Var  8  (sec) 

Pref 

Subject  1 

33.8 

29.7 

27.7 

26.5 

8, 7, 6, 5 

Subject  2 

24.5 

22.5 

19.8 

18.3 

8, 7, 6, 5 

Subject  3 

18.7 

18.7 

18.4 

18.7 

7, 8, 6, 5 

Subject  4 

29.6 

29.5 

27.8 

28.4 

7, 8, 6, 5 

Subject  5 

16.7 

16.6 

14.2 

8, 7, 6, 5 

Subject  6 

23.3 

20.2 

16.9 

17 

7, 8, 6, 5 

Subject  7  * 

63.4 

56.5 

41.7 

8, 7, 6, 5 

Statistics  with  7  Subjects 

Average 

30.0  ±  1.9 

27.7  ±  1.6 

23.9  ±  1.1 

23.5  ±  1.0 

8, 7, 6, 5 

Median 

24.7 

23.9 

19.9 

19.5 

8, 7, 6, 5 

Statistics  with  Subjects  1,2, 3, 4, 5, 

6 

Average  (6) 

23.4  ±  1.1 

22.3  ±  0.8 

8, 7, 6, 5 

Median  (6) 

23.5 

21.3 

18.5 

18.8 

7, 8, 6, 5 

For  most  subjects  there  was  a  significant  improvement  in  time  as  the  visual  size  increased 
from  “Small”  to  “Medium” ,  and  “Medium”  to  “Large.”  There  was,  however,  no  significant 
improvement  when  the  visual  size  was  increased  from  “Large”  to  “Ex-large.”  Both  the  mean 
time  and  the  median  time  showed  this  trend.  Variation  “Small”  had  a  mean  completion 
time  of  23.4  ±  1.1  seconds.  Variation  “Medium”  had  a  mean  time  of  22.3  ±  0.8  seconds. 
Variation  “Large”  had  a  mean  time  of  20.4  ±  0.8  seconds.  Variation  “Ex-large”  had  a  mean 
time  of  20.1  ±  0.7  seconds.  The  median  times  were  23.5,  21.3,  18.8,  and  18.5  seconds  for 
“Small”,  “Medium”,  “Ex-large”,  and  “Large”  respectively.  Figure  5-8  shows  the  boxplot 
for  the  time  results  of  Experiment  6. 

Table  5-16  show's  the  mean  error  results  for  Experiment  6.  The  error  counts  did  not 
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Boxplot  of  Time  Results  for  Experiment  6:  Decreased  Visual  Scaling  (6  Subjects) 


Figure  5-8:  Boxplot  of  the  time  results  for  Experiment  6:  Decreasing  Visual  Scale.  Variation 
5  (“Small”):  mean  =  23.4  ±  1.1,  median  =  23.5.  Variation  6  (“Medium”):  mean  =  22.3  ± 
0.8,  median  =  21.3.  Variation  7  (“Large”):  mean  =  20.4  ±  0.8,  median  =  18.5.  Variation 
8  (“Ex-large”):  mean  =  20.1  ±  0.7,  median  =  18.8.  (sec) 
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vary  linearly  with  increasing  visual  size.  Variation  “Large”  had  the  lowest  error  count  at 
138  ±  20,  followed  by  Variation  “Medium”  with  148  ±  26.  Variation  “Ex-large”  had  the 
next  lowest  error  count,  176  ±  32.  Variation  “Small”  had  the  highest  error  count,  191  ±  39. 
The  median  values  for  the  error  count  had  a  trend  different  from  the  mean  value.  Variation 
“Small”  had  the  lowest  at  62  counts,  followed  by  variation  “Medium”  with  73  counts,  then 
by  variation  “Large”  with  75  counts.  Variation  “Ex-large”  had  the  highest  median  error 
at  86  counts.  Figure  5-9  shows  the  boxplot  for  the  error  results  of  Experiment  6.  There 
were  many  trials  in  which  subjects  performed  without  any  error.  Variation  Large  had  the 
most  trials  without  errors,  26  trials.  Variation  “Medium”  had  23  no  error  trials,  followed 
by  Variation  “Ex-large”  with  19  trials,  and  Variation  “Small”  with  14  trials. 

There  were  no  wall  crossing  errors  in  this  experiment  since  the  subjects  were  told  to 
minimize  wall  contact. 


Table  5.16:  Mean  Error  Performance  for  Experiment  6:  Decreasing  Visual  Scale 


Small 

Medium 

Large 

Ex-large 

Var  5 

Var  6 

Var  7 

Var  8 

Pref 

Subject  1 

185 

270 

189 

173 

8, 5, 7, 6 

31 

12 

12 

12 

7, 6, 8, 5 

Subject  3 

300 

274 

541 

wm\ 

Subject  4 

190 

205 

ifcHhfcii 

Subject  5 

74 

67 

94 

129 

6, 5, 7, 8 

Subject  6 

68 

47 

51 

76 

6, 7, 5, 8 

Subject  7  * 

3 

33 

19 

4 

5, 8, 7, 6 

Statistics  with  7  Subjects 

Average 

164  ±  34 

131  ±  22 

121  ±  18 

151  ±  29 

7, 6, 8, 5 

Median 

33 

58 

22 

45 

7, 5, 8, 6 

no  error  trials 

26 

34 

39 

32 

7, 6, 8, 5 

Statistics  with  Subjects  1,2, 3, 4, 5, 6 

Average  (6) 

191  ±  39 

148  ±  26 

138  ±  20 

176  ±  32 

7, 6, 8, 5 

62 

73 

75 

86 

5, 6, 7, 8 

no  error  trials 

14 

23 

26 

19 

7, 6, 8, 5 

Table  5-17  shows  the  preference  ranking  for  the  variations  in  Experiment  6.  The  subjects 
consistently  preferred  lager  visual  mazes  to  smaller  visual  mazes.  In  the  comments  given  in 
the  post-session  survey,  subjects  said,  “the  larger  mazes  were  easier. ..there  was  more  room 
to  move  in.” 
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Boxplot  of  Error  Results  for  Experiment  6:  Decreased  Visual  Scaling  (6  Subjects) 


Variation 

Figure  5-9:  Boxplot  of  the  error  results  for  Experiment  6:  Decreasing  Visual  Scale.  Variation 
1  (“Small”):  mean  =  191  ±  39,  median  =  62.  Variation  2  (“Medium”):  mean  =  14S  ± 
26,  median  =  73.  Variation  3  (“Large”):  mean  =  138  ±  20,  median  =  75.  Variation  4 
(“Ex-large”):  mean  =  176  ±  32,  median  =  86.  (counts) 


Table  5.17:  Preference  Rankings  for  Experiment  6:  Decreasing  Visual  Scale 


Var 

Sub  1 

Sub  2 

Sub  3 

Sub  4 

Sub  5 

Sub  6 

Sub  7 

Total 

5)  Small 

16 

7 

23 

14 

33 

16 

18 

127 

6)  Medium 

18 

21 

EQHI 

19 

35 

23 

24 

167 

7)  Large 

24 

32 

warn 

30 

35 

29 

29 

210 

8)  Ex-large 

31 

35 

mwm ■ 

29 

35 

35 

34 

231 

Preference 

8, 7, 6, 5 

8, 7, 6, 5 

8, 7,6,5 

7, 8, 6, 5 

8, 7, 6, 5 

8, 7, 6, 5 
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5.5  Cursor  Paradigms 


Table  5-18  lists  the  characteristics  for  each  of  the  cursor  paradigms  used  in  Experiment  7. 

Table  5.18:  Cursor  Paradigms 


Analogy 

“mouse” 

“magnifying 

lens” 

“tele¬ 

video” 

“RC  car” 

Paradigm 

i 

2 

3 

4 

Visual  Cursor 

moving 

static 

Visual  Environment 

static 

moving 

moving 

static 

Haptic  Cursor 

moving 

moving 

static 

static 

Haptic  Environment 

static 

static 

Control  Type 

Position 

Control 

Position 

Control 

Force 

Control 

Force 

Control 

5.5.1  Experiment  7:  Position  and  Force  Control  Cursor  Paradigms 

In  this  experiment,  the  subjects  were  told  to  complete  mazes  as  quickly  as  possible  without 
passing  through  the  walls.  Table  5-19  shows  the  time  required  to  complete  the  paradigms 
presented  in  Experiment  7. 


Table  5.19:  Mean  Time  Performance  for  Experiment  7:  Cursor  Control  Paradigms 


Para  1  (sec) 

Para  2  (sec) 

Para  3  (sec) 

Para  4  (sec) 

Pref. 

Subject  1 

24 

38 

1,2, 4, 3 

Subject  2 

13.7 

36.3 

34.9 

1,2, 4, 3 

Subject  3 

14 

21 

28 

24 

1,2, 4, 3 

Subject  4 

10.4 

11.7 

36.5 

31.4 

1,2, 4, 3 

Subject  5 

7.4 

9.9 

31.1 

28.3 

1,2, 4, 3 

Subject  6 

9.6 

13.4 

25.1 

24.3 

1,2, 4, 3 

Subject  7 

10.4 

17.4 

34.3 

1,2, 4, 3 

Average 

10.8  ±  0.3 

16.1  ±  0.7 

33.4  ±  1.0 

27.3  ±  0.6 

1,2, 4, 3 

9.7 

15.1 

32.2 

1,2, 4, 3 

Position  control  paradigms  required  significantly  less  time  to  complete  than  the  force 
control  paradigms.  The  “mouse”  paradigm  required  the  least  amount  of  time  to  complete, 
10.8  ±  0.3  seconds,  followed  by  the  “magnifying  lens”  paradigm  with  16.1  ±  0.7  seconds. 
Subjects  completed  the  position  control  paradigms  in  less  than  half  the  time  of  the  force 
control  paradigms.  The  “RC  car”  paradigm  averaged  27.3  ±  0.6  seconds  to  complete.  The 
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“tele- video”  paradigm  had  the  highest  mean  completion  time,  33.4  ±  1.0  seconds.  The  trend 
for  the  median  values  of  the  four  paradigms  was  consistent  with  the  trend  for  the  mean 
values.  The  median  times  were  9.7,  15.1,  27.1  and  32.2  seconds  for  the  “mouse”  paradigm, 
“magnifying  lens”  paradigm,  “RC  car”  paradigm,  and  “tele-video”  paradigm  respectively. 
Figure  5-10  shows  the  boxplot  of  the  time  results  for  the  cursor  control  paradigm  experiment. 


Boxplot  of  Time  Results  for  Cursor  Control  Paradigms 


Figure  5-10:  Boxplot  of  the  time  results  for  Experiment  7:  Cursor  Control  Paradigm. 
Paradigm  1  (“mouse”):  mean  =  10.8  ±  0.3,  median  =  9.7.  Paradigm  2  (“lens”):  mean  = 
16.1  ±  0.7,  median  =  15.1.  Paradigm  3  (“video”):  mean  =  33.4  ±  1.0,  median  =  32.2. 
Paradigm  4  (“RC  car”):  mean  =  27.3  ±  0.6,  median  =  27.1.  (sec) 

There  were  zero  wall  errors  for  the  “mouse”  Paradigm,  zero  for  the  “magnifying  lens” 
Paradigm,  53  for  the  “tele-video”  Paradigm,  and  35  for  the  “RC  car”  Paradigm.  Since 
the  force  control  cursor  paradigm  was  new  to  the  subjects,  some  found  it  quite  difficult 
to  control;  hence,  the  numerous  wall  crossing  errors.  Many  subjects  also  found  the  “tele¬ 
video”  paradigm  most  difficult  of  the  paradigm  since  visual  scene  was  not  stationary.  Several 
subjects  found  the  moving  visual  scene  distracting. 

Table  5-20  shows  the  preference  rankings  of  the  cursor  paradigms  for  Experiment  7. 
Overall,  the  subjects  found  the  “mouse”  paradigm  easiest,  then  the  “lens”,  “RC  car”, 
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and  finally  the  “tele-video”  paradigm.  This  was  the  exact  same  order  as  that  of  time 
performance.  Some  of  the  subjects  found  “tele-video”  to  be  easier  than  “RC  car”  because 
it  had  an  analogous  movement  of  cursor  and  environment.  Most  subjects  found  “RC  car” 
to  be  easier  because  they  liked  a  stationary  background. 


Table  5.20:  Preference  Rankings  for  Experiment  7:  Cursor  Control  Paradigms 


Sub  1 

Sub  2 

Sub  3 

Sub  4 

Sub  5 

Sub  6 

Sub  7 

Avg 

1)  Mouse 

73 

73 

74 

74 

75 

72 

74 

74 

2)  Lens 

59 

60 

60 

66 

53 

42 

57 

57 

3)  Video 

43 

45 

46 

38 

31 

34 

42 

40 

4)  RC  car 

31 

45 

44 

49 

39 

36 

54 

43 

Preference 

1,2, 3, 4 

1,2, 3, 4 

1,2, 3, 4 

1,2, 4, 3 

1,2, 4, 3 

1,2, 4, 3 

1,2, 4, 3 

1,2, 4, 3 

Given  the  two  position  control  and  two  force  control  cursor  paradigms,  subjects  found 
position  control  to  be  much  easier  than  force  control.  The  ratings  found  the  “mouse” 
paradigm  to  be  easiest  over  all,  followed  by  “magnifying  lens”.  The  force  control  cursor 
paradigms  were  equally  rated,  with  some  subjects  preferring  “tele-video”  paradigm  and 
some  subjects  preferring  “RC  car”  Paradigm.  Most  found  “tele-video”  paradigm  easier 
for  the  force  control  because  the  correlation  of  the  cursor  to  the  environment  was  more 
consistent  among  the  visual  and  haptic  displays.  This  paradigm  had  a  stationary  haptic 
cursor  and  a  stationary  visual  cursor. 
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Chapter  6 


Discussion 


6.1  Visual-Haptic  Size  Experiments 

6.1.1  Accuracy  and  Speed 

Subjects  performed  best  and  felt  most  at  ease  when  presented  with  a  small  haptic  workspace 
corresponding  to  a  large  visual  workspace.  This  allowed  them  to  take  advantage  of  their  fine 
motor  control  skills.  The  large  visual  presentation  let  them  have  an  enhanced  view  of  their 
haptic  workspace.  However,  when  presented  with  such  a  workspace,  the  wall  did  not  feel  very 
stiff.  This  was  due  to  the  effect  of  visual  dominance.  The  walls  felt  stiffer  when  the  subjects 
were  presented  with  a  small  visual  workspace  coupled  with  a  large  haptic  workspace.  Time 
to  completion  -was  inversely  related  to  the  wall  contact  error  count.  Subjects  were  able  to 
complete  the  maze  faster  if  they  stayed  in  contact  with  the  wall  more.  Many  used  a  strategy 
of  having  the  wall  as  a  guide  to  go  straight,  then  turn  in  the  appropriate  direction  when 
they  struck  an  opposing  wall. 

6.1.2  Sensory  Feedback  Experiments 

Subjects  could  complete  the  mazes  faster  when  they  were  given  both  visual  and  haptic 
cues.  When  deprived  of  one  of  these  sensory  modes,  subjects  performed  better  when  given 
only  haptic  cues.  The  results  of  these  experiments  showed  the  importance  of  having  haptic 
feedback  in  completing  a  task.  On  average,  there  was  a  26%  decrease  in  performance  from 
having  both  visual  and  haptic  feedback  to  having  only  haptic  feedback.  There  was  a  61% 
decrease  in  performance  from  having  both  visual  and  haptic  feedback  to  having  only  visual 
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feedback.  Figures  6-1  through  6-4  show  the  boxplot  trends  of  the  various  sensory  feedback 
modes  for  Variations  1  though  4,  respectively.  Performance  was  much  more  consistent  when 
subjects  were  given  both  visual  and  haptic  feedback.  The  variance  of  the  quartile  data  was 
much  less  for  the  visual  and  haptic  feedback  than  for  either  of  the  haptic  feedback  only  or 
the  visual  feedback  only.  Haptic  feedback  only  had  the  largest  variance  in  quartile  data 
since  subjects  sometimes  became  lost  in  the  maze  and  required  more  time  to  complete  the 
navigation. 

Subjects  preferred  and  performed  best  on  a  small  visual  and  haptic  size  when  given  both 
visual  and  haptic  feedback.  Subject  preferred  a  large  visual  and  haptic  size,  but  performed 
best  on  a  small  visual  and  haptic  size  for  the  haptic  feedback  only  experiment.  Subjects 
preferred  and  performed  best  on  a  large  visual  size  and  a  small  haptic  size  for  the  visual 
feedback  only  experiment.  It  is  unexpected  that  the  subject’s  performance  and  preference 
for  a  certain  variation  would  change  given  different  sensory  feedback  modes. 


Boxplot  for  Variation  1 :  Visual-Haptic,  Haptic,  and  Visual  Feedback 


Figure  6-1:  Boxplot  of  Variation  1  for  Sensory  Feedback  Experiments.  Visual  and  Haptic: 
mean  =  6.9  ±  0.2,  median  =  6.9.  Haptic  Only:  mean  =  8.3  ±  0.4,  median  =  7.7.  Visual 
Only:  mean  =  10.1  ±  0.4,  median  =  9.5.  (sec) 
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Boxplot  for  Variation  2:  Visual-Haptic,  Haptic,  and  Visual  Feedback 


Figure  6-2:  Boxplot  of  Variation  2  for  Sensory  Feedback  Experiments.  Visual  and  Haptic: 
mean  =  7.2  ±  0.2,  median  =  7.2.  Haptic  Only:  mean  =  9.7  ±  0.5,  median  =  9.3.  Visual 
Only:  mean  =  10.7  ±  0.3,  median  =  10.6.  (sec) 
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Boxplot  for  Variation  3:  Visual-Haptic,  Haptic,  and  Visual  Feedback 


Figure  6-3:  Boxplot  of  Variation  3  for  Sensory  Feedback  Experiments.  Visual  and  Haptic: 
mean  =  6.0  ±  0.2,  median  =  5.7.  Haptic  Only:  mean  =  7.6  ±  0.4,  median  =  7.6.  Visual 
Only:  mean  =  9.0  ±  0.3,  median  =  8. 4 (sec) 
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Boxplot  for  Variation  4:  Visual-Haptic,  Haptic,  and  Visual  Feedback 


Figure  6-4:  Boxplot  of  Variation  4  for  Sensory  Feedback  Experiments.  Visual  and  Haptic: 
mean  =  6.1  ±  0.2,  median  =  5.6.  Haptic  Only:  mean  =  7.4  ±  0.4,  median  =  6.7.  Visual 
Only:  mean  =  8.9  ±  0.3,  median  =  8.7(sec) 
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6.2  Visual  Scaling  Experiments 


The  results  of  experiments  1,  2,  3,  and  4,  showed  that  subjects  preferred  and  performed  best 
when  given  a  large  visual-haptic  size  ratio.  A  second  set  of  experiments  was  designed  to 
test  the  extent  of  this  phenomenon.  Would  performance  always  continue  to  improve,  would 
performance  level  off  at  some  critical  ratio,  or  would  performance  degrade  at  a  critical  ratio? 
In  the  visual  scaling  experiments,  subjects  were  presented  with  various  visual  workspace 
sizes  and  a  single  haptic  workspace  size. 

When  subjects  were  trained  on  the  largest  visual  size  in  the  series,  their  performance 
improved  as  the  visual  workspace  size  increased.  A  significant  improvement  in  time  to 
completion  was  seen  as  the  visual-haptic  ratio  increased  from  1:1  to  1.5:1  and  from  1.5:1  to 
2:1.  However,  the  improvement  in  completion  time  was  not  as  significant  as  the  visual-haptic 
size  ratio  increased  from  2:1  to  2.5:1.  This  showed  a  limit  in  the  effect  of  visual  dominance. 
Once  the  visual  size  was  larger  than  twice  the  haptic  size,  the  subject’s  performance  no 
longer  improved. 

However,  when  subjects  were  trained  on  the  smaller  visual  size  in  the  series,  their  overall 
performance  did  not  improve  steadily.  Some  subjects  improved  steadily,  but  other  subjects 
improved  and  then  leveled  off.  Still  others  improved  and  then  regressed.  This  also  showed 
that  there  was  a  limit  in  improvement  of  performance  when  visual  display  was  magnified. 
There  was  a  perception  of  finer  motor  control  when  the  field  of  view  was  magnified. 

6.3  Cursor  Paradigm  Experiment 

All  the  subjects  found  a  position  controlled  cursor  to  be  much  easier  to  use  than  a  force 
controlled  cursor.  All  the  subjects  also  preferred  the  mouse  paradigm.  This  might  have 
been  because  all  the  subjects,  being  university  students  who  work  on  the  computer  a  fair 
amount  and  use  the  mouse,  had  already  been  pre-trained  in  the  use  of  this  position  control 
paradigm  and  therefore  preferred  it.  The  subjects  preferred  the  paradigms  in  the  following 
order  “Mouse” ,  “Magnifying  lens” ,  “RC  car” ,  and  “Tele-video” .  Their  performance  was  in 
the  same  order  as  their  preference. 

Most  subjects  required  50%  more  time  to  complete  the  maze  for  “magnifying  lens” 
paradigm  than  “mouse”  paradigm.  Although  both  were  position  control,  the  paradigm 
with  a  higher  visual-haptic  consistency  was  easier  to  complete.  Only  one  subject  required 
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only  10%  more  time. 

Force  control  paradigms  required  significantly  more  time  to  complete  than  position 
control  paradigms.  Most  subjects  required  at  least  twice  as  much  time,  some  as  much  as  6 
times  that  of  Paradigm  1.  Subjects  found  force  control  paradigms  more  difficult,  possibly 
due  the  fact  that  it  was  a  more  complex  method  of  control.  Not  only  was  precision  of 
motion  needed,  but  precision  of  force  was  also  needed. 

Some  subjects  found  it  easier  if  the  visual  presentation  and  the  haptic  presentation  were 
consistent.  In  other  words,  if  the  visual  cursor  was  moving,  it  was  easier  to  complete  the 
maze  if  the  haptic  cursor  was  also  moving.  Also  if  the  visual  cursor  was  stationary,  it  was 
easier  if  the  haptic  cursor  was  stationary.  Some  subjects,  however,  found  it  easier  if  the 
background  was  always  stationary. 

6.4  Ergonomics 

Several  subjects  complained  about  discomfort  in  their  wrists.  The  forearm,  wrist,  and  hand 
must  be  correctly  supported  in  haptic  tasks,  especially  if  the  task  required  the  human  to  use 
motor  control  skills  for  a  sustained  amount  of  time.  Though  the  arm  rest  helped  to  raise  the 
arm  and  hand  to  the  correct  position  for  some  subjects,  it  was  not  ideal  for  all  subjects.  It 
also  did  not  compensate  for  motion.  A  stationary  arm  rest  was  useful  for  stationary  tasks, 
but  an  armrest  that  can  rotate  and  move  would  be  better  for  dynamic  tasks. 
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Chapter  7 


Conclusions 

7.1  MAGIC  Toolkit 

The  MAGIC  Toolkit  is  very  useful  application  that  allows  a  user  to  quickly  and  easily  create 
a  virtual  scene.  The  library  files  also  allow  a  user  to  integrate  the  virtual  scene  into  the 
user’s  specific  application.  Improvements  to  the  MAGIC  Toolkit  include  the  advancement 
to  3-dimensional  graphics,  the  addition  of  material  properties,  and  the  addition  of  dynamic 
properties.  Though  2-dimensional  graphics  suffice  for  some  applications,  3-dimensional 
graphics  are  necessary  for  a  more  immersive  effect.  Other  material  properties  that  could 
be  added  to  the  object’s  characteristics  include  texture,  mass,  and  viscosity.  The  addition 
of  virtual  mechanisms  such  as  pin  joints  or  ball  joints  would  allow  dynamic  interactions 
between  the  objects  in  the  VE. 


7.2  Interaction  Paradigms 

An  effect  of  scaling  which  may  prove  useful  in  making  walls  or  surfaces  seem  more  stiff  is  to 
have  the  visuals  of  a  smaller  scale  than  the  haptics.  This  fact  was  discovered  when  several 
subjects  commented  on  the  fact  that  it  is  much  easier  to  pass  through  the  wall  on  some 
variations  than  others.  When  looking  over  the  traces  of  the  subjects,  the  tendency  to  pass 
through  the  walls  was  much  stronger  for  visuals  that  were  larger  than  the  haptics.  The 
corollary  to  this  is  that  smaller  visual  scales  will  imitate  a  stiffer  wall.  What  we  already 
know  of  visual  dominance  with  the  spring  experiment,  supports  this  effect  [Srinivasan  et. 
al.,  1996]. 
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Performance  results  conflicted  with  what  the  subjects  perceived.  Though  subjects  found 
the  “mouse”  paradigm  to  be  the  easiest  to  use,  an  improvement  could  be  seen  in  the 
completion  time  of  the  maze  for  all  cursor  paradigms.  This  means  that  a  person  can  be 
trained  to  use  a  specific  paradigm  effectively.  The  four  different  cursor  paradigms  defined 
in  this  can  be  used  for  a  large  variety  of  tasks. 

7.3  Future  Work 

For  some  tasks,  only  a  small  workspace  is  needed.  Future  haptic  interfaces  can  be  developed 
with  this  fact  in  mind.  The  implementation  of  haptic  interfaces  can  take  advantage  of  force 
control  interactions  to  gain  a  larger  workspace.  Further  studies  on  the  effectiveness  of  using 
force  control  need  to  be  investigated. 

The  MAGIC  Toolkit  can  be  used  to  develop  sensory  tasks  for  rehabilitative  medicine. 
Improvements  in  the  MAGIC  Toolkit  include  the  addition  of  3-dimensional  graphics,  more 
material  properties,  and  dynamic  behavior  of  objects.  This  will  allow  for  a  broader  appli¬ 
cation  of  multimodal  environments. 


88 


Appendix  A 


Instructions  for  Experiments 


Experiment  1: 

Objective:  To  complete  the  maze  as  accurately  as  possible  without  touching  the  walls 

Procedure:  Before  each  trial  you  will  be  given  a  blank  screen  with  a  circle  in  the  mid¬ 
dle.  There  are  two  cursors  representing  your  current  location.  The  large  yellow  cursor  is 
a  forward  projection  of  your  location.  The  small  white  cursor  is  a  right  projection  of  your 
location.  You  need  to  center  both  of  these  cursors  in  the  circle  before  you  start.  This  is  to 
make  sure  you  start  at  the  same  spot  for  every  trial. 

You  hold  the  stylus  in  your  hand  as  you  would  a  pencil  and  rest  your  forearm  in  the 
arm  rest.  Once  you  have  aligned  the  cursor  you  can  press  any  key  to  start. 

A  red  maze  will  appear.  There  will  also  be  a  green  starting  block  and  a  black  finishing 
block.  Move  the  Stylus  forward  until  you  feel  the  front  wall.  To  start,  move  to  the  left  and 
touch  the  green  block  gently.  The  block  will  turn  light  green.  Once  you  let  go,  the  timer 
will  start  and  While  you  are  going  through  the  maze,  the  only  surface  you  should  touch 
is  the  front  surface.  Try  to  minimize  contact  with  any  other  surface.  Once  you  touch  the 
black  finishing  block  you  will  be  finished  with  that  trial. 

There  are  a  total  of  62  trials.  You  will  be  asked  to  rank  several  of  the  trials  on  a  scale 
of  1  to  5  on  ease  of  completing  the  maze  half  way  through  the  session.  (5  being  the  easiest, 
1  being  the  hardest) 

Experiment  2: 

Objective:  To  complete  the  maze  as  quickly  as  possible.  You  can  use  the  wall  of  the 
maze  to  help  guide  you  along. 

Procedure:  Before  each  trial  you  will  be  given  a  blank  screen  with  a  circle  in  the  mid¬ 
dle.  There  are  two  cursors  representing  your  current  location.  The  large  yellow  cursor  is 
a  forward  projection  of  your  location.  The  small  white  cursor  is  a  right  projection  of  your 
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location.  You  need  to  center  both  of  these  cursors  in  the  circle  before  you  start.  This  is  to 
make  sure  you  start  at  the  same  spot  for  every  trial. 

You  hold  the  stylus  in  your  hand  as  you  would  a  pencil  and  rest  your  forearm  in  the 
arm  rest.  Once  you  have  aligned  the  cursor  you  can  press  any  key  to  start. 

A  red  maze  will  appear.  There  will  also  be  a  green  starting  block  and  a  black  finishing 
block.  Move  the  Stylus  forward  until  you  feel  the  front  wall.  To  start,  move  to  the  left  and 
touch  the  green  block  gently.  The  block  will  turn  light  green.  Once  you  let  go,  the  timer 
will  start  and  While  you  are  going  through  the  maze,  please  stay  in  contact  with  the  front 
surface.  You  can  use  the  other  wall  surfaces  to  guide  you  through  the  maze.  Once  you 
touch  the  black  finishing  block  you  will  be  finished  with  that  trial. 

There  are  a  total  of  45  trials.  You  will  be  asked  to  rank  several  of  the  trials  on  a  scale 
of  1  to  5  on  ease  of  completing  the  maze  after  the  5th  trial. 

1  =  very  hard 

2  =  hard 

3  =  not  too  hard,  not  too  easy 

4  =  easy 

5  =  very  easy 

Experiment  3: 

Objective:  To  complete  the  maze  as  quickly  as  possible.  You  can  use  the  wall  of  the 
maze  to  help  guide  you  along.  There  will  not  be  a  cursor  indicating  your  position  along  the 
maze. 

Procedure:  Before  each  trial  you  will  be  given  a  blank  screen  with  a  circle  in  the  mid¬ 
dle.  There  are  two  cursors  representing  your  current  location.  The  large  yellow  cursor  is 
a  forward  projection  of  your  location.  The  small  white  cursor  is  a  right  projection  of  your 
location.  You  need  to  center  both  of  these  cursors  in  the  circle  before  you  start.  This  is  to 
make  sure  you  start  at  the  same  spot  for  every  trial. 

You  hold  the  stylus  in  your  hand  as  you  would  a  pencil  and  rest  your  forearm  in  the 
arm  rest.  Once  you  have  aligned  the  cursor  you  can  press  any  key  to  start. 

A  red  maze  will  appear.  There  will  also  be  a  green  starting  block  and  a  black  finishing 
block.  Move  the  Stylus  forward  until  you  feel  the  front  wall.  To  start,  move  to  the  left 
and  touch  the  green  block  gently.  The  block  will  turn  light  green.  Once  you  let  go,  the 
timer  will  start  and  While  you  are  going  through  the  maze,  please  stay  in  contact  with  the 
front  surface.  You  can  use  the  other  wall  surfaces  to  guide  you  through  the  maze.  Once 
you  touch  the  black  finishing  block  you  will  be  finished  with  that  trial.  There  will  not  be  a 
cursor  to  show  you  your  exact  location.  Use  the  feel  of  the  maze  to  guide  you  through. 

There  are  a  total  of  45  trials.  You  will  be  asked  to  rank  several  of  the  trials  on  a  scale 
of  1  to  5  on  ease  of  completing  the  maze  after  the  5th  trial. 

1  =  very  hard 

2  =  hard 
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3  =  not  too  hard,  not  too  easy 

4  =  easy 

5  =  very  easy 

Experiment  4: 

Objective:  To  complete  the  maze  as  quickly  as  possible.  Follow  the  path  through  the 
maze.  You  may  touch  the  walls  of  the  maze,  but  try  not  to  pass  through  the  walls.  There 
will  be  a  visual  cursor  indicating  your  position  along  the  maze.  You  will  not  have  any  force 
feedback  from  the  walls  of  the  maze. 

Procedure:  Before  each  trial  you  will  be  given  a  blank  screen  with  a  circle  in  the  mid¬ 
dle.  There  are  two  cursors  representing  your  current  location.  The  large  yellow  cursor  is 
a  forward  projection  of  your  location.  The  small  white  cursor  is  a  right  projection  of  your 
location.  You  need  to  center  both  of  these  cursors  in  the  circle  before  you  start.  This  is  to 
make  sure  you  start  at  the  same  spot  for  every  trial. 

You  hold  the  stylus  in  your  hand  as  you  would  a  pencil  and  rest  your  forearm  in  the 
arm  rest.  Once  you  have  aligned  the  cursor  you  can  press  any  key  to  start. 

A  red  maze  will  appear.  There  will  also  be  a  green  starting  block  and  a  black  finishing 
block.  Move  the  Stylus  forward  until  you  feel  the  front  wall.  To  start,  move  to  the  left  and 
touch  the  green  block  gently.  The  block  will  turn  light  green.  Once  you  let  go,  the  timer 
will  start  and  While  you  are  going  through  the  maze,  please  stay  in  contact  with  the  front 
surface.  Once  you  touch  the  black  finishing  block  you  will  be  finished  with  that  trial.  There 
will  be  a  cursor  to  show  you  your  exact  location  It  is  OK  to  visually  touch  the  walls  of  the 
maze,  but  Try  not  to  pass  through  the  walls  of  the  maze. 

There  are  a  total  of  45  trials.  You  will  be  asked  to  rank  several  of  the  trials  on  a  scale 
of  1  to  5  on  ease  of  completing  the  maze  after  the  5th  trial. 

1  =  very  hard 

2  =  hard 

3  =  not  too  hard,  not  too  easy 

4  =  easy 

5  =  very  easy 

Experiment  5  &  6 

Objective:  To  complete  the  maze  as  accurately  as  possible  without  touching  the  walls. 

Procedure:  Before  each  trial  you  will  be  given  a  blank  screen  with  a  circle  in  the  mid¬ 
dle.  There  are  two  cursors  representing  your  current  location.  The  large  yellow  cursor  is 
a  forward  projection  of  your  location.  The  small  white  cursor  is  a  right  projection  of  your 
location.  You  need  to  center  both  of  these  cursors  in  the  circle  before  you  start.  This  is  to 
make  sure  you  start  at  the  same  spot  for  every  trial. 

You  hold  the  stylus  in  your  hand  as  you  would  a  pencil  and  rest  your  forearm  in  the 
arm  rest.  Once  you  have  aligned  the  cursor  you  can  press  any  key  to  start. 
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A  red  maze  will  appear.  There  will  also  be  a  green  starting  block  and  a  black  finish¬ 
ing  block.  Move  the  Stylus  forward  until  you  feel  the  front  wall.  To  start,  move  to  the 
left  and  touch  the  green  block  gently.  The  block  will  turn  light  green.  Once  you  let  go, 
the  experiment  will  start  and  While  you  are  going  through  the  maze,  the  only  surface  you 
should  touch  is  the  front  surface(floor  of  the  maze).  Try  to  minimize  contact  with  any  other 
surface.  Once  you  touch  the  black  finishing  block  you  will  be  finished  with  that  trial. 

There  are  a  total  of  62  trials.  You  will  be  asked  to  rank  several  of  the  trials  on  a  scale 
of  1  to  5  on  ease  of  completing  the  maze  half  way  through  the  session. 

1  =  very  hard 

2  =  hard 

3  =  not  too  hard,  not  too  easy 

4  =  easy 

5  =  very  easy 

Experiment  7: 

Objective:  To  complete  the  maze  as  accurately  as  possible  without  passing  through  the 
walls.  You  can  use  the  walls  as  a  guide,  but  be  careful  not  to  apply  too  much  force,  since 
it  is  possible  to  pass  through  the  walls. 

Procedure:  Before  each  trial  you  will  be  given  a  blank  screen  with  a  circle  in  the  mid¬ 
dle.  There  are  two  cursors  representing  your  current  location.  The  large  yellow  cursor  is 
a  forward  projection  of  your  location.  The  small  white  cursor  is  a  right  projection  of  your 
location.  You  need  to  center  both  of  these  cursors  in  the  circle  before  you  start.  This  is  to 
make  sure  you  start  at  the  same  spot  for  every  trial. 

You  hold  the  stylus  in  your  hand  as  you  would  a  pencil  and  rest  your  forearm  in  the 
arm  rest.  Once  you  have  aligned  the  cursor  you  can  press  any  key  to  start. 

A  colored  maze  will  appear.  There  will  also  be  a  green  starting  block  and  a  black  fin¬ 
ishing  block.  Move  the  Stylus  forward  until  you  feel  the  front  wall.  To  start,  move  to  the 

left  and  touch  the  green  block  gently.  The  block  will  turn  light  green.  Once  you  let  go, 
the  experiment  will  start.  While  you  are  going  through  the  maze,  stay  in  contact  the  front 
surface(floor  of  the  maze).  Once  you  touch  the  black  finishing  block  you  will  be  finished 
with  that  trial. 

There  are  four  different  types  of  maze  interfaces.  Each  of  the  4  interfaces  are  color  coded 
to  help  you  distinguish  them. 

In  some  mazes  you  will  control  your  location  by  where  you  move, 

in  other  mazes  you  will  control  your  location  by  the  force  you  apply. 

In  some  mazes  your  motion  will  change  the  location  of  the  cursor, 
in  other  mazes  you  motion  will  change  the  location  of  the  background. 

There  are  a  total  of  72  trials.  You  will  be  asked  to  rank  several 
of  the  trials  on  a  scale  of  1  to  5  on  ease  of  completing  the  maze. 

1  =  very  hard 
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2  =  hard 

3  =  not  too  hard,  not  too  easy 

4  =  easy 

5  =  very  easy 
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Appendix  B 


Boxplots 


Boxplot  for  Time  Results  of  Experiment  1 :  Accuracy  (7  Subjects) 


Figure  B-l:  Boxplot  of  the  time  results  for  Experiment  1:  Accuracy.  Variation  1:  mean  = 
21.5  ±  1.3,  median  =  17.4  .  Variation  2:  mean  =  23.2  ±  1.5,  median  =  19.7  .  Variation  3: 
mean  =  18.2  ±  1.0,  median  =  15.9.  Variation  4:  mean  =  19.5  ±  1.1,  median  =  16.3.  (sec) 
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Boxplot  for  Error  Counts  of  Experiment  1 :  Accuracy  (7  Subjects) 


Figure  B-2:  Boxplot  of  the  error  results  for  Experiment  1:  Accuracy.  Variation  1:  mean  = 
389  ±  50,  median  =  209.  Variation  2:  mean  =  445  ±  54,  median  =  227.  Variation  3:  mean 
=  470  ±  56,  median  =  302  .  Variation  4:  mean  =  414  ±  52,  median  =  230.  (counts) 


Boxplot  of  Time  Results  for  Experiment  2:  Visual  and  Haptic  Feedback  (5  Subjects) 


Figure  B-3:  Boxplot  of  the  time  results  for  Experiment  2:  Speed,  Visual  and  Haptic  Feed¬ 
back.  Variation  1:  mean  =  9.3  ±  0.7,  median  =  7.1.  Variation  2:  mean  =  9.8  ±  0.8, 
median  =  7.3.  Variation  3:  mean  =  8.0  ±  0.6,  median  =  6.0.  Variation  4:  mean  =  8.2  ± 
0.6,  median  =5.8.  (sec) 
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Boxplot  of  Time  Results  for  Experiment  3:  Haptic  Feedback  Only  (5  Subjects) 


Figure  B-4:  Boxplot  of  the  time  results  for  Experiment  3:  Speed,  Haptic  Feedback  Only. 
Variation  1:  mean  =  10.2  ±  0.7,  median  =  8.2.  Variation  2:  mean  =  12.0  ±  0.9,  median 
=10.2.  Variation  3:  mean  =  8.6  ±  0.4,  median  =  8.3.  Variation  4:  mean  =  8.6  ±  0.5, 
median  =  7.6.  (sec) 
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Boxplot  of  Time  Results  for  Experiment  4:  Visual  Feedback  Only  (5  Subjects) 


Figure  B-5:  Boxplot  of  the  time  results  for  Experiment  4:  Speed,  Visual  Feedback  Only. 
Variation  1:  mean  =  12.9  ±  0.6,  median  =  12.2  .  Variation  2:  mean  =  13.8  ±  0.7,  median 
=12.5.  Variation  3:  mean  =  11.6  ±  0.6,  median  =  10.9.  Variation  4:  mean  =  11.5  ±  0.5, 
median  =  10.8.  (sec) 
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Boxplot  of  Time  Results  for  Experiment  6:  Decreased  Visual  Scaling  (7  Subjects) 


Figure  B-6:  Boxplot  of  the  time  results  for  Experiment  6:  Decreased  Visual  Scaling.  Varia¬ 
tion  5  (“Small”):  mean  =  30  ±  1.9,  median  =  24.7.  Variation  6  (“Medium”):  mean  =  27.7 
±  1.6,  median  =  23.9.  Variation  7  (“Large”):  mean  =  23.9  ±  1.1,  median  =  19.9.  Variation 
8  (“Ex-large”):  mean  =  23.5  ±  1.0,  median  =  19.5.  (sec) 
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Variation 


Figure  B-7:  Boxplot  of  the  error  results  for  Experiment  6:  Decreased  Visual  Scaling.  Vari¬ 
ation  5  (“Small”):  mean  =  164  ±  34,  median  =  33.  Variation  6  (“Medium”):  mean  =  131 
±  22,  median  =  58.  Variation  7  (“Large”):  mean  =  121  ±  18,  median  =  22.  Variation  8 
(“Ex-large”):  mean  =  151  ±  29,  median  =  45.  (counts) 
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